Hi
The custom transformer
SchemaSetter will do most of the work for you, just insert it right after the FeatureMerger. You will probably have to set up a couple of "Prefixes to exclude", e.g. "fme_,multi_,csv_,shape_" to avoid including them in your output shape file.
If you need more fine grained control, you might also want to look up
this article about how to define a schema based on a list attribute.
David
Hi
The custom transformer
SchemaSetter will do most of the work for you, just insert it right after the FeatureMerger. You will probably have to set up a couple of "Prefixes to exclude", e.g. "fme_,multi_,csv_,shape_" to avoid including them in your output shape file.
If you need more fine grained control, you might also want to look up
this article about how to define a schema based on a list attribute.
David
Thank you so much for this!
In this partiucular case I have no control of what attributes will be in the incoming csv file, attributes are generally generated dynamically in the workspace, this transformer (after excluding several prefixes as you have mentioned) did what I needed.
The article you have linked is known to me, however there are listed only methods where you statically insert which attributes you want to expose / write.
Hi
The custom transformer
SchemaSetter will do most of the work for you, just insert it right after the FeatureMerger. You will probably have to set up a couple of "Prefixes to exclude", e.g. "fme_,multi_,csv_,shape_" to avoid including them in your output shape file.
If you need more fine grained control, you might also want to look up
this article about how to define a schema based on a list attribute.
David
However it seems there are several attributes missing, I must be doing something wrong in the writer part.
The problem might be this: The Writer will only check for the list on the first feature of each Feature Type
Seeing that the list on the first feature is created but is not filled, because the first feature did not have all of the attributes present.
Each feature in this case can have different attributes. Maybe that is causing problems?
However it seems there are several attributes missing, I must be doing something wrong in the writer part.
The problem might be this: The Writer will only check for the list on the first feature of each Feature Type
Seeing that the list on the first feature is created but is not filled, because the first feature did not have all of the attributes present.
Each feature in this case can have different attributes. Maybe that is causing problems?
Yes, not having all the attributes present on the first feature will be a bit more challenging.
Here's a small hack to make the feature with the most attributes arrive at the writer first:
Before the writer, add a PythonCaller with the following code
import fme
import fmeobjects
def FeatureProcessor(feature):
count = len(feature.getAllAttributeNames())
feature.setAttribute('_number_of_attributes', count)
In the PythonCaller, expose the attribute name _number_of_attributes
Then add a Sorter and sort descending (numeric) on _number_of_attributes.
The feature that has the maximum number of attributes should then be the one that defines the output schema.
Note: there are some situations where the above might not suffice, notably where no single feature contains all possible attributes (those cases might need more advanced scripting), so be sure to test before deploying.
David
Yes, not having all the attributes present on the first feature will be a bit more challenging.
Here's a small hack to make the feature with the most attributes arrive at the writer first:
Before the writer, add a PythonCaller with the following code
import fme
import fmeobjects
def FeatureProcessor(feature):
count = len(feature.getAllAttributeNames())
feature.setAttribute('_number_of_attributes', count)
In the PythonCaller, expose the attribute name _number_of_attributes
Then add a Sorter and sort descending (numeric) on _number_of_attributes.
The feature that has the maximum number of attributes should then be the one that defines the output schema.
Note: there are some situations where the above might not suffice, notably where no single feature contains all possible attributes (those cases might need more advanced scripting), so be sure to test before deploying.
David
Yeah, with almost complete certainity I can already say that none of the features are going to contain all of the attributes. I'll remember this workaround. :)
I have found a way to make the Schemasetter work. I have output the features based on their respective object types, where I can presume that each object in specific object group is going to have the same set of attributes (e.g. roads, buildings,...). When I do this, I can make the dynamic writer take schema from first feature.
All your answers had been very helpful, thank you very much. :)
Yeah, with almost complete certainity I can already say that none of the features are going to contain all of the attributes. I'll remember this workaround. :)
I have found a way to make the Schemasetter work. I have output the features based on their respective object types, where I can presume that each object in specific object group is going to have the same set of attributes (e.g. roads, buildings,...). When I do this, I can make the dynamic writer take schema from first feature.
All your answers had been very helpful, thank you very much. :)
Hi @drakez,
> I can presume that each object in specific object group is going to have the same set of attributes.
If so, this article may also be helpful
Destination Schema is Derived from a Lookup Table
Takashi
Yeah, with almost complete certainity I can already say that none of the features are going to contain all of the attributes. I'll remember this workaround. :)
I have found a way to make the Schemasetter work. I have output the features based on their respective object types, where I can presume that each object in specific object group is going to have the same set of attributes (e.g. roads, buildings,...). When I do this, I can make the dynamic writer take schema from first feature.
All your answers had been very helpful, thank you very much. :)
This link actually is really useful too. The attribute description and actual values are coming from database table, my csv file was just a sample of data I am working with. I cannot share the database connection (obviously), I was just stuck at this part where I had to expose the attributes for shapefile writer.
The tables in this database are really specific, I tried deriving destination schema from database tables, but each table has only part of the whole information (there is a table that has information about attributes, however the info about data type of each attribute is stored in another table...), so before coming to the csv I had included here I had to join several tables. I guess that I have to be happy that it works the way @david_r had suggested.
Yeah, with almost complete certainity I can already say that none of the features are going to contain all of the attributes. I'll remember this workaround. :)
I have found a way to make the Schemasetter work. I have output the features based on their respective object types, where I can presume that each object in specific object group is going to have the same set of attributes (e.g. roads, buildings,...). When I do this, I can make the dynamic writer take schema from first feature.
All your answers had been very helpful, thank you very much. :)
For info and/or inspiration: I have done something similar myself, reading the oracle metadata tables (all_tab_columns in particular) to dynamically create the attribute list using Python before sending it to a writer as the output schema definition.
Using the same technique, it is possible to join several tables together in e.g. a SQLCreator and use a PythonCaller to dynamically assemble the attributes{} list based on the query result. You can create the list attribute in one "track" in your workspace and read/process your data in another, then merge them together at the end of your processing, just before hitting the writer.
David
