Question

Passing published parameter into Schemamapper within a custom transformer

  • 3 September 2019
  • 4 replies
  • 1 view
D

Hi Everyone,

 

I tried to make use of the schemamapper transformer to create a new output excel spreadsheet which takes a customised schema csv file as an input. The problem is that the schema file sits within a random folder (date time stamp temp folder name - e.g. 20190903100000) each time the workbench run. (PS: the randomised folder is created at the beginning of the workflow and being passed through as an attribute value).Due to the randomised temp folder name which i tried to pass into the schemamapper as the dataset, I am not able to pass it through. I then tried to create a custom transformer where i can pass the temp folder name via attribute value > published parameter for the custom transformer. The new problem I have now is that the custom transformer does not appear to be able to pick up the attribute value (which has become a published parameter) in the dataset field within the schemamapper.

It always throws an error shows "ERROR |CSV reader: Failed to open file '@Value(working_folder)Schema.csv' for reading. Please ensure that the file exists and you have sufficient privileges to read it"

Can anyone advise what would be the best method to approach this?

Note that I am aware that I can initiate another workbench which works perfectly but I would like to keep it in a single workbench.

Please find attached sample workbench.

Thanks you everyone for your assistance.

4 replies

sigtill
Rank
Badge +21

Hi! Whenever I use paths within Custom Transformers I try to add them as attributes - for instance _fullfilename. Also use the "Single Output-port" if the schema is not important. Make sure the fullfilename is available by double clicking the green Inputbox.

Hope this helps @doni.tan

The real SigTill
D

Hi! Whenever I use paths within Custom Transformers I try to add them as attributes - for instance _fullfilename. Also use the "Single Output-port" if the schema is not important. Make sure the fullfilename is available by double clicking the green Inputbox.

Hope this helps @doni.tan

Hi @sigtill,

Thanks you for your quick response. Sorry for not being up front, the issue is more specific on schemaMapper transformer rather than other transformers. I tried what you mentioned above on featureReader and it works perfectly. But if i tried it on the schemaMapper transformer, it will throw the same error.

I have actually created a sample workbench and attached to the initial question this time.

But thanks for your assistance.

jdh
Rank
Badge +22

The SchemaMapper transformer can only take a hardcoded value or a published parameter as the Dataset Parameter. Wrapping it in a custom transformer doesn't change this behaviour, it just allows you to type the hardcoded value, rather than using the file browser GUI.

 

 

This means that the SchemaMapper is not resolving the value of the attribute, it's looking for a file called '@Value...'

 

 

The only way I see this working is if you used a scripted private parameter to figure out the Dataset based on the TempFolder parameter, and use that in the SchemaMapper.

 

 

You would probably want to include the creating the directory portion of the workspace in this parameter rather than using the systemCaller.

 

 

I would also suggest creating an idea to allow the SchemaMapper to use attributes as input parameters. Though I don't know how technologically feasible that is.
D

The SchemaMapper transformer can only take a hardcoded value or a published parameter as the Dataset Parameter. Wrapping it in a custom transformer doesn't change this behaviour, it just allows you to type the hardcoded value, rather than using the file browser GUI.

 

 

This means that the SchemaMapper is not resolving the value of the attribute, it's looking for a file called '@Value...'

 

 

The only way I see this working is if you used a scripted private parameter to figure out the Dataset based on the TempFolder parameter, and use that in the SchemaMapper.

 

 

You would probably want to include the creating the directory portion of the workspace in this parameter rather than using the systemCaller.

 

 

I would also suggest creating an idea to allow the SchemaMapper to use attributes as input parameters. Though I don't know how technologically feasible that is.

Hi @jdh, thanks you for the input. I am hoping to find out if anyone has had any alternative solution such as creating alternate flow to mimic how schemaMapper works. In this case, it will does not require scripting which require in depth interpretation on the workflow.

Reply


Community Stats

28,670
Posts
109,366
Replies
36,531
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.