Files can be read by Parents but not by its Children?

Question

Dear Community,

Last year I created a very large FME Workspace that became even larger in time as new feature requests came in while there was no time to restructure the FME Workspace. At the end the FME Workspace had more than 1300 Transformers and FME Workbench had trouble to open it. (The file was 25 Mb large.) Moreover development work became slow as I had to wait about 10 seconds every time I clicked within FME Workbench. Fortunately I convinced my client to restructure the FME Workspace and now there is time to do just that.

With an upgrade of the FME Server from 2016 to 2018 it becomes possible to make use of Parent-Child-relationships. Hence no additional engines had to be bought and that made it possible to reshape one FME Workspace in two FME Workspaces. Unfortunately I was surprised that if I ran the Child FME Workspace alone (which is in that case by definition no Child) a File Geodatabase could be read. (See job 274 in the screenshot below.) If I used however a Parent FME Workspace that submits a Child FME Job the File Geodatabase could not be read! (See job 275 and 276 in the screenshot.)

The only difference I could see here is that the queue has a different name. (Speaking with the FME Server Administrator he told me that there exist in reality only one queue within the organization.)

After doing multiple tests using test FME Workspaces an explanation could be that:

* Files seem to become locked if they are first read in the Parent and can hence no more be read by the Child FME Workspace;

* Copying Files does not seem to work either: the source file or destination file seem to become locked as well;

* Even using a FeatureReading and reading only the file names (including their paths) seem to lock the map in which the files are located.

Does anyone know how this problem can be solved?

(The file names are a variable in the upload and are normally deduced by reading the files in a FME Server Shared Folder.)

I found a workaround that is good enough for my assignment: also using Customized Transformers the size of the FME Workspace can be significantly reduced in a similar way as you would split the FME Workspaces in more than one. Apparently the size of a FME Workspace are more correlated on the number of connections (and amount of information that need to be transferred through) than on the number of Transformers.

I am looking forward to hear from you.

Jochgem

Page 1 / 1

An additional remark.

A test using parallel processing, in which in the first process starting with a Creator, a Feature Reader reading a folder to know the URL to be read, a Published Parameter reading the URL, a second process also starting with a Creator, with a Decelerator that is used to have a delay of for example 10 seconds and a FMEJobSubmitter that executes the Child Workspace using an extra Published Parameter to transfer the URL, DOES seem to work.

It comprises a Decelerator that delays the whole process which makes the solution sub optimal and subject to the performance of the machine the process is executed with.

Using a similar principal using a VariableSetter and VariableRetriever did not have the desired results.

Best, Jochgem

Reply

jochgem · Answer

An additional remark.A test using parallel processing, in which in the first process starting with a Creator, a Feature Reader reading a folder to know the URL to be read, a Published Parameter reading the URL, a second process also starting with a Creator, with a Decelerator that is used to have a delay of for example 10 seconds and a FMEJobSubmitter that executes the Child Workspace using an extra Published Parameter to transfer the URL, DOES seem to work.It comprises a Decelerator that delays the whole process which makes the solution sub optimal and subject to the performance of the machine the process is executed with.  Using a similar principal using a VariableSetter and VariableRetriever did not have the desired results.Best, Jochgem

Reply

Community Stats

Sign up

An FME Account is required to contribute

Login to the community

An FME Account is required to contribute

Scanning file for viruses.

This file cannot be downloaded