Solved

Workspace Runner success/failure, what if the target ESRI FGDB is locked, retaining each FME translation log

  • 19 October 2016
  • 4 replies
  • 6 views

Badge +7

Not so much a question as bringing together a few things...

I'm using a Workspace Runner to call a Workspace which writes to an ESRI File Geodatabase (FGDB). In testing, I had the FGDB Feature Class open in ArcCatalog which caused the translation to fail because the target was locked. Also, the Workspace Runner was reporting success, even though the Workspace it was calling failed.

There are two things here. Firstly, "Wait for job to complete" was set to "No" which meant that the Workspace Runner would report a success even if the Workspace it called failed. "No" enables you to run concurrent Workspaces which is something I did not want to do because I need to process a series of input datasets in order, not in parallel. The solution is to set "Wait for job to complete" to "Yes":

http://docs.safe.com/fme/2016.1/html/FME_Desktop_Documentation/FME_Transformers/Transformers/workspacerunner.htm

Secondly, the locking issue can often be solved using @david_r's suggestion of truncating the target Feature Class instead of dropping it:

https://knowledge.safe.com/questions/1827/arcsde-geodatabase-locks.html

Finally, if the Workspace Runner calls a Workspace more than once, the translation log of the first of that Workspace will be overwritten by subsequent runs. I'm going to try the LogMessageStreamer as suggested by @mark2catsafe:

https://knowledge.safe.com/questions/30394/capture-translation-log.html

http://docs.safe.com/fme/2016.1/html/FME_Desktop_Documentation/FME_Transformers/Transformers/logmessagestreamer.htm

icon

Best answer by takashi 20 October 2016, 01:33

View original

4 replies

Userlevel 2
Badge +17

Hi @tim_wood, regarding the log file, you can also create a published parameter linking to Log File parameter (one of built-in Workspace Parameters) and pass unique log file paths for each run to the workspace through this parameter.

I hope that this idea will be realized: Add "Log File" parameter to WorkspaceRunner

In addition, when you run the workspace from the command line, you can also pass the log file path with a command line option called "LOG_FILENAME".

Badge +7

Hi @tim_wood, regarding the log file, you can also create a published parameter linking to Log File parameter (one of built-in Workspace Parameters) and pass unique log file paths for each run to the workspace through this parameter.

I hope that this idea will be realized: Add "Log File" parameter to WorkspaceRunner

In addition, when you run the workspace from the command line, you can also pass the log file path with a command line option called "LOG_FILENAME".

Thanks. I like the sound of that.

 

I tried Log Message Streamer but it only works when the translation was a success! When the translation failed because the FGDB feature class was locked, no features were output from the Log Message Streamer. I tried various values for the parameters with no success. Is it something to do with "Limitations: Messages outside the lifetime of the [Log Message Streamer] transformer cannot be captured." Has the lifetime of the transformer ended once the features are at the Writer?

 

I've set up a Published Parameter with Type "Filename", Configuration "Extension|*.log".

 

I tried to set a Default Value of $(WORKSPACE_NAME).LOG but when I try to run the Workspace it fails with an error saying "Undefined macro `WORKSPACE_NAME' dereferenced in file `<PATH REDACTED>\\wb-xlate-1476975783613_2540'. If the parameter is not supplied, I want the log file name to default to the FMW name, whatever it's called at the time (e.g. I might rename the FMW).

 

For now I'll go with making the parameter not optional and leaving Default Value empty which will force the user to pick a log file (hopefully the one with the same name as the FMW).

 

 

Badge +1

I would like the exit status of the workbench to be returned to the workspace runner. Then I can respond to the failure to not continue.

In my case I run a process that splits up the data into 80 regions. I can run 7 at a time in parallel and then merge the regions at the end. But only if they are all successful. I too sometimes get a write lock which is not returned as an error. Could I try running a python exit script? After all something gets returned if some errors in the workbench trigger a failure signal.

Badge

I would like the exit status of the workbench to be returned to the workspace runner. Then I can respond to the failure to not continue.

In my case I run a process that splits up the data into 80 regions. I can run 7 at a time in parallel and then merge the regions at the end. But only if they are all successful. I too sometimes get a write lock which is not returned as an error. Could I try running a python exit script? After all something gets returned if some errors in the workbench trigger a failure signal.

sorry to dig out the old post, but have you managed to return exit code 1 if some of the parallel jobs fail?

Reply