Skip to main content
Question

How to generate an output schema on the fly based on a transformer output


Forum|alt.badge.img
Is there a way to generate an output schema on the fly based on a transformer output ?

 

Suppose I dynamically rename an attribute with today's date as a suffix.

 

How can my writer's schema reflect this ever changing attribute name ?

11 replies

  • November 15, 2012
You can use a Concatenator to combine everything you want into the filename (including constants or attribute values) and call the concatenated attribute FILENAME (or something similar.   You can then use the fanout option on the reader to fanout based on FILENAME.

 

 

I do this to create shapefile backups by date.

  • November 15, 2012
Sorry, mis-read your question.  Where is the delete button!?

Forum|alt.badge.img
  • November 23, 2012
You can download a small sample here:

 

http://www.sendspace.com/file/pew12w

 

 

The trick is to use a writer with a dynamic schema and use TFS as the source schema. A scripted python parameter modify the TFS schema before the workspace reads it and returns the current date used to name the attribute. A python caller is then used to rename the attribute according to the date returned py the scripted python parameter.

 


Forum|alt.badge.img
  • Author
  • November 23, 2012
Brilliant!

 

 

Merci!!!

Forum|alt.badge.img
  • Author
  • November 23, 2012
larry wrote:
You can download a small sample here:

 

http://www.sendspace.com/file/pew12w

 

 

The trick is to use a writer with a dynamic schema and use TFS as the source schema. A scripted python parameter modify the TFS schema before the workspace reads it and returns the current date used to name the attribute. A python caller is then used to rename the attribute according to the date returned py the scripted python parameter.

 

Brilliant!

 

 

Merci!!!

helmoet
Forum|alt.badge.img+8
  • November 10, 2015
Use a SchemaSetter and a Dynamics writer. Dynamic Properties: - Schema Sources from originating dataset - Feature Type name from whatever discriminating attribute there is - Schema Definition from the same attribute

fmelizard
Contributor
Forum|alt.badge.img+17
  • Contributor
  • November 10, 2015
jerrod wrote:
You can use a Concatenator to combine everything you want into the filename (including constants or attribute values) and call the concatenated attribute FILENAME (or something similar.   You can then use the fanout option on the reader to fanout based on FILENAME.

 

 

I do this to create shapefile backups by date.

@jerrod -- even though your comment didn't answer this question, it is still is a very valid and good point and useful for anyone wanting files/tables named after the current date.


gio
Contributor
Forum|alt.badge.img+15
  • Contributor
  • November 11, 2015
Well still can't expose dynamic attributes within one workspaceautomatically.

 

 

I use a Filewriter ot create a schemafile from the output and have it start a secondary workspace to read the file. Parameters are set tot "none" After first creating one, you can change this to none.

 

THi way u don't need to expose anything manually.

 

There are a couple of examples of this technique on th forum. (mostly mine..;)

 

 

 

Sometimes it is possible to transfer some logic (miostly simple) form a workspace to a python or tcl script wich is then used as a start-up. Creating the required s chema. Mostly this is not possible, namely with attributes created on the fly based on complex logic. (else you might as well do the entire proces in python or tcl or whatever)

 


fmelizard
Contributor
Forum|alt.badge.img+17
  • Contributor
  • November 11, 2015
gio wrote:
Well still can't expose dynamic attributes within one workspaceautomatically.

 

 

I use a Filewriter ot create a schemafile from the output and have it start a secondary workspace to read the file. Parameters are set tot "none" After first creating one, you can change this to none.

 

THi way u don't need to expose anything manually.

 

There are a couple of examples of this technique on th forum. (mostly mine..;)

 

 

 

Sometimes it is possible to transfer some logic (miostly simple) form a workspace to a python or tcl script wich is then used as a start-up. Creating the required s chema. Mostly this is not possible, namely with attributes created on the fly based on complex logic. (else you might as well do the entire proces in python or tcl or whatever)

 

@gio -- this may not completely address your workflow needs, but @brianpont 's tutorial on Dynamic Workflows https://knowledge.safe.com/articles/1050/dynamic-workflow-tutorial-introduction.html is worth a careful look. In particular, the last sub-tutorial in there on schemas from list attributes may provide some insights.

Aside: We know we still need to work further on making it easier to manipulate and create schemas within workbench and so that work is ongoing. But in the meantime between Brian's techniques and scripting, you can accomplish amazingly powerful transformations.


takashi
Influencer
  • November 11, 2015
fmelizard wrote:

@gio -- this may not completely address your workflow needs, but @brianpont 's tutorial on Dynamic Workflows https://knowledge.safe.com/articles/1050/dynamic-workflow-tutorial-introduction.html is worth a careful look. In particular, the last sub-tutorial in there on schemas from list attributes may provide some insights.

Aside: We know we still need to work further on making it easier to manipulate and create schemas within workbench and so that work is ongoing. But in the meantime between Brian's techniques and scripting, you can accomplish amazingly powerful transformations.

@daleatsafe, @brianpont's tutorial is very helpful, but the interface of writer feature type properties has been changed in FME 2015.1. Although the "deriving from list" method described in the tutorial still can be used, users can configure more easily in 2015.1+. Hope the tutorial will be updated (add descriptions for 2015.1+).

Also have a look at related PR#65579.


brianatsafe
Safer
Forum|alt.badge.img+11
  • Safer
  • November 16, 2015
takashi wrote:

@daleatsafe, @brianpont's tutorial is very helpful, but the interface of writer feature type properties has been changed in FME 2015.1. Although the "deriving from list" method described in the tutorial still can be used, users can configure more easily in 2015.1+. Hope the tutorial will be updated (add descriptions for 2015.1+).

Also have a look at related PR#65579.

Thank you @takashi. I wanted to let you know that we plan to update the Dynamic tutorials shortly to integrate the new UI (2015.1 +) and show of the new FeatureReader capability (Schema port).


Reply


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings