Skip to main content
Open

Parameterize SchemaMapper's Dataset Parameters

Related products:Transformers
  • September 14, 2016
  • 9 replies
  • 150 views

takashi
Celebrity

The scenario: There are multiple databases which store datasets for the same feature types, but table names and column names are vary for each. The mission is to migrate the multiple databases to new multiple datasets which have common schema (i.e. common feature type names, common attribute names).

Other than data tables, every database has two mapping tables - one defines the mapping rule between table names and destination common feature type names, another defines the mapping rule between column names and destination common attribute names for each feature type. Fortunately, schema of the mapping tables are common to every database.

If it was possible to set user parameters or feature attributes to the SchemaMapper's Dataset parameters (including database connection parameters), a workspace containing a SchemaMapper could do the task with batch processing. However, the Dataset parameters cannot be parameterized in the current SchemaMapper, so you need to set the connection parameters manually for each database.

In fact, I resolved this with a workflow containing Python scripts which perform schema mapping for each database based on the mapping tables, but I thought it would be easier if the SchamaMapper's Dataset parameters could be parameterized.

9 replies

hsamor
Forum|alt.badge.img
  • October 10, 2016

I'd like the database connection parameters to be parameterized, very much!


nic_ran
Contributor
Forum|alt.badge.img+20
  • Contributor
  • June 19, 2017

The name of the file (spreadsheet) used for schema mapping should be able to be provided via a published parameter.


Forum|alt.badge.img+6

I ran into this same issue last week. However please note that you CAN assign a parameter, but it's a little bit backwards: go to Navigator, expand the transformer, then right-click on "Dataset" and link to a user parameter. I also submitted this to support, who gave me this idea link. Thanks.

 


Forum|alt.badge.img
  • January 18, 2019

I would also find it very useful if the source dataset for the SchemaMapper could be parameterized. That way if you are using multiple spread sheets all with the same schema you would set a parameter each time you switch spreadsheets instead of having to have a SchemaMapper transformer for each spreadsheet.


doni.tan
Participant
  • Participant
  • September 6, 2019

I have encountered a similar issue where i wanted to dynamically populate the dataset due to the potential different schema csv file and its location across different run of the workbench. Unfortunately there is no other way than initiating another workbench as schemaMapper only receive public parameter or the static schema full file name (file path and filename with file extension).

It will be come very powerful if we are able to have this feature available especially on process automation in micro services.

https://knowledge.safe.com/questions/97949/passing-published-parameter-into-schemamapper-with.html?childToView=98195#answer-98195

 


fallingdog
Contributor
Forum|alt.badge.img+6
  • Contributor
  • February 18, 2020

Yes, please!


j.botterill
Influencer
Forum|alt.badge.img+60
  • Influencer
  • February 17, 2021

I have a number of projects using schema mapper, formats for excel with 5 sheets or MS access tables. A published parameter for dataset would be super helpful


ebygomm
Evangelist
Forum|alt.badge.img+51
  • Evangelist
  • April 19, 2023

This saved my day, which FME is doing its best to ruin.


LizAtSafe
Safer
Forum|alt.badge.img+21
  • Safer
  • February 8, 2024
The following idea has been merged into this idea:

All the votes have been transferred into this idea.