Question

Transferring list of input files from one workspace to another ?

10 years ago
26 November 2013
8 replies
3 views

gispro
15 replies

Hi,

I have the task of setting up a flexible transformation, first in workbench, later on an FME Server.

The input is a list of file paths from a view in an Oracle database, and based on a "category number" for each file, the relevant files of a given category should be converted by one out of multiple distinct workspaces. I.e. a 1-n "master-slaves" setup.

Note that the "features" in the "master" workspace is non-geom records from a alphanumeric view. I.e. not the features that need to be processed in the "slave" workspaces.

I have the master workspace setup, reading the Oracle list, splitting the output based on the category number, collecting the path names into a list, concatenating this into a semicolon separated text, and transferring this concatenated text as a single feature/attribute to a workspacerunner. I'm applying the separated text parameter to the source parameter of the input dataset in the called "slave" workspace.

But when I run the "master" workspace, it looks like the "slave" workspaces aren't run. And no output is produced by them.

I'm obviously doing something wrong, and I fear the I'm utilizing a couple of wrong assumptions. So here are a couple of basic questions to start with:

- Will the WorkspaceRunner run once for each submitted feature (non-geom) ?

- Will the WorkspaceRunner transfer texts to source input path lists in the called workspace, and will they be recognized as input specs in the called workspace ?

- How should a path name list be separated and escaped to work ?

Cheers.

8 replies

The files to be converted as DGN files, by the way.

Userlevel 4

Hi,

the WorkspaceRunner will indeed be called once for every feature that enters.

If your master workspace have a list of files to process you will have to convert it to a string before passing it on to the child workspace's published parameter. The format is usually

"filename"<space>"filename"<space>"filename"...etc

You do not need to escape the paths as long as you have double quotation marks around each item, like above.

Just be aware that if you have a very large number of files to process, the resulting string may be too long to pass as a published parameter. If that is the case, consider saving the list of files to a temporary text file and then passing that filename to the next workspace.

To debug this functionality consult the log files, they are indespensable.

David

I just re-visited an example from April 2008 by Mark Ireland ("Polygon Building"). It shows a parameter with a value of "&path_windows".

Is this a hidden string concatenator, and is this still relevant and the way to do it in a 2013 version ?

Besides, it uses a "PATH reader" which I cannot find in 2013.

Thanks David,

Is "<space>" a valid value for the separator ?

Userlevel 4

Hi,

you must replace <space> with a proper space character.

The reader in question is surely the "Directory and File Pathnames" Reader.

I do not know what the "&path_window" is.

David

Hello David,

I have this specific portion of my processing located inside a embedded/custom transformer, and wanted the path separator as a published argument. This unfortunately doesn't work with a space character, so I had to revert to hardcoding it. That does work.

I found that one obstruction I was facing was the licensing. Each workspace requires a separate license, and I was running on a floating license, which then didn't allow me to start the other workspace.

As I was going to upload it to a FME Server eventually, I did that, and changed my WorkspaceRunner til a JobSubmitter. Unfortunately the job exits with an error, which I haven't had a chance to investigate, but at least it looks like the arguments are getting transferred correctly :-)

The total request string is almost 17k long, and the longest path argument is almost 5000 characters long. Is either of these a problem ?

Thanks for your help.

Userlevel 4

Hi,

I don't know the hard limits when using the JobSubmitter, but in general terms I find 17K quite a lot.

You'll either have to experiment (e.g. by trying with just one or two file paths) or ask Safe support if you need a concrete answer, I think.

David

Thanks David,

I've submitted a support ticket about the possible limitations and explanation of the error.

Quite strange error response too: resultSucces=false but serviceSucces=true.

Transferring list of input files from one workspace to another ?

8 replies

Reply

Community Stats

Reply

Community Stats

Sign up

An FME Account is required to contribute

Login to the community

An FME Account is required to contribute

Scanning file for viruses.

This file cannot be downloaded