Skip to main content
  • EN
I am spawning the execution of an FME workspace from (running FME.exe from .NET System.Diagnositcs.START() ) works fine and I can catch the errors and the completion.

 

 

Now the issue is that this is acutally a long running job (I am using WebMapTiler to create about 5K - 25K PNG map tiles). It can take up to 10 minutes to run.

 

 

I would like to provide status messages to the spawning .NET program (let's caller it spawner.exe) and the workspace that it spawns MakeTiles.fmw. I am using FME Desktop 2013 (Budget will not handle FME server, since we would need hundreds of engines. Besides it is overkill for what the sponsor wants).

 

 

I want messages of the format: "Created 456 of 5,674 tiles" "Estimated completion in 5.32 minutes"

 

 

I can think of three methods, but I need more information on how to do this (or which one is the best).

 

 

1. Polling the log file

 

I can have spawner.exe do a 2-5 (sec.) repeating read/parse of the FME log file. I see the log does provide a hard number for how many tiles it is going to create, and it it should be possible to scan the text for how many are done.

 

 

PROS: It probably will work, and I don't need to learn anything

 

CONS: Kludgy

 

    fragile (any change to the syntax of log messages will kill it

 

    high overhead (parsing, greping, etc. of the log text

 

    I hope I can open for read while this is being written.

 

    I have to do a sum of the counts of the lines written

 

    

 

2. HTTPFetcher

 

My Spawener.exe is actually a stand-alone WCF data server. So it is easy to open up a localhost port (e.g 4326) and get messages from the HTTPFetcher transform that would have arguments telling me what the current tile count and total tile count are. I like this idea the best, but I have no idea how to get this info out of the WebMapTiler or anything downstream of it (Expression Evaluator, StringConcatantor, or PNGRaster writer).

 

 

PROS: clean, simple, robust

 

CONS: Need information on how to implement

 

 

3. Polling a file

 

If I can get the current tile & total tiles to output - from the WebMapTiler, then I could conceivably write a file (and keep re-writing it) with the text "Created 456 of 5,674 tiles". I can do the time estimates from that. But I have no idea how to write such a text file, nor how to get the informaiton for this file. Nor how to keep re-writing it.

 

 

PROS: pretty clean, and robust

 

CONS: Still requires a poll read-file loop from the server

 

      Lots of open issues

 

      

 

      

 

Any ideas for me. Something I left out?
Hi,

 

 

you could use a PythonCaller to send periodic messages to the calling application. There are a lot of modules for inter-process communication bundled with the standard Python distribution.

 

 

This solution ought to give you a lot of flexibility.

 

 

David
Yes, I could, but I am still missing the key information from the WebTileMapper as to which tile it has finished, and how many to go. (actually, since FME does lazy eval - I really need this from the PNGRASTER writer, since I don't know if the WebTileMapper knows how many tiles the writer has finished!

 

 

 

Hi,

 

 

I do not think the transformers have any notion about the activity of the writers in FME. I think you would have to do some of the work yourself here as FME can't do this out of the box.

 

 

You could try one PythonCaller at the start of your process to calculate the number of tiles to create, which you could store as a global (Python) variable. It should be fairly easy to calculate the number of tiles (e.g. using a BoundingBoxReplacer and an ExpressionEvaluator).

 

 

Another PythonCaller just before the writer could count the passing features and send a periodic signal to the calling application, containing the total number of tiles from the first PythonCaller as well as the counter value.

 

 

David
What about another idea. I do know how many NTF images go in (with the resolution and size). Is there a formula I could use to get the number of

 

256x256 PNG files I would get out?

 

 

 

If so, I might be able to make .NET OS Directory calls to see how many files are currently in the output folder (from the C# program, not from FME). But that is where I want status reports.

 

 

 

What do you think of that?

Reply


Community Stats

30,157
Posts
114,473
Replies
37,802
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.