Hi everyone, I'm facing an issue in my workflow: when I process two shapefiles with the same name, the Feature Reader (in Generic mode) interprets them as a single file. This causes an error in my Join. Is there a way to handle this so both shapefiles are read separately?
Solved
Problem in Feature reader
+5- Contributor
Best answer by francisco_1988
I assume that the two source Shapefile datasets have the same file name (feature type name) but have different schema and you need to read schema feature for each file with the same FeatureReader.
A possible way is:
- create a custom transformer containg a FeatureReader and publish the Group By parameter to make it a group-based transformer,
- in the main workflow, you can set an initiator's attribute that stores source Shapefile path to the Group By parameter of the custom transformer to read schema separately for each dataset, even if the different datasets have an idential file name.
To publish the Group By parameter in a custom transformer definition, right-click on Transformer Parameters/Group By and select "Create User Parameter", on the Navigator of the canvas tab for defining the custom transformer. See also the screenshot below.
Hi Takashi
Thank you!!!
This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
Hi
I think the direct reason for the rejection at the FeatureJointer is a Join On value - i.e. the value of "fme_feature_type_name" attribute was missing. Firstly check the messages in the log and documents related to the failing situation - the Help on FeatureJointer in this case.
Configuration / Output Ports / <Rejected>
Invalid features from any input port. Features will be rejected if a Join On value is missing or null.
And, if you are attempting to configure dynamic schema for the writer based on the Schema feature read from the FeatureReader, it might be a possible way to send the Schema feature into writer feature type directly, instead of merging it to data features.
This article contains an example for using Schema feature read from FeatureReader to configure dynamic schema.
Extracting Data within a Clipping Polygon using FeatureReader
+5- Contributor
Hi
I think the direct reason for the rejection at the FeatureJointer is a Join On value - i.e. the value of "fme_feature_type_name" attribute was missing. Firstly check the messages in the log and documents related to the failing situation - the Help on FeatureJointer in this case.
Configuration / Output Ports / <Rejected>
Invalid features from any input port. Features will be rejected if a Join On value is missing or null.
And, if you are attempting to configure dynamic schema for the writer based on the Schema feature read from the FeatureReader, it might be a possible way to send the Schema feature into writer feature type directly, instead of merging it to data features.
This article contains an example for using Schema feature read from FeatureReader to configure dynamic schema.
Extracting Data within a Clipping Polygon using FeatureReader
The issue occurred because the fme_feature_type_name attribute was missing. This happens because I configured a dynamic schema in the writer based on the schema read by the FeatureReader. However, I have a question: after the FeatureReader, my workflow needs to continue, but in your approach, it seems a writer would be placed right after it. Could you explain better how to proceed without interrupting the workflow?
The workflow shown in the screenshot I pasted in the previous post is just an example in a case where you don't need to transform data features output from the <Generic> port at all.
If you would like to configure dynamic writer schema with the schema feature output from the <Schema> port, you can connect it to the writer feature type directly.
Regarding the data features, you can insert any transformers between the <Generic> port and the destination writer feature type in order to transform them, if necessary.
+5- Contributor
The workflow shown in the screenshot I pasted in the previous post is just an example in a case where you don't need to transform data features output from the <Generic> port at all.
If you would like to configure dynamic writer schema with the schema feature output from the <Schema> port, you can connect it to the writer feature type directly.
Regarding the data features, you can insert any transformers between the <Generic> port and the destination writer feature type in order to transform them, if necessary.
Hi Takashi!
I'll briefly explain how my workflow works and how I relate the Schema and Generic outputs.
I start by reading an Excel spreadsheet.
Each row in the spreadsheet contains the path to a shapefile.
I use a FeatureReader to dynamically read each shapefile based on those paths.
My goal is to extract the schema (attribute structure) of each shapefile in order to generate an XML at the end of the process, which acts like a data dictionary.
Then, I use a FeatureJoiner to join the features from the <Generic> port with those from the <Schema> port of the FeatureReader.
The problem arises when I have two shapefiles with the same name (for example, bairros.shp in two different folders).
In those cases, FME generates only one schema, and this causes an error in the FeatureJoiner, because it expects a 1:1 match between schema and feature data.
I assume that the two source Shapefile datasets have the same file name (feature type name) but have different schema and you need to read schema feature for each file with the same FeatureReader.
A possible way is:
- create a custom transformer containg a FeatureReader and publish the Group By parameter to make it a group-based transformer,
- in the main workflow, you can set an initiator's attribute that stores source Shapefile path to the Group By parameter of the custom transformer to read schema separately for each dataset, even if the different datasets have an idential file name.
To publish the Group By parameter in a custom transformer definition, right-click on Transformer Parameters/Group By and select "Create User Parameter", on the Navigator of the canvas tab for defining the custom transformer. See also the screenshot below.
Why not inspect features with Visual/Data Preview and Feature/Record Information before writing them into a destination dataset?
+5- Contributor
- Best Answer
I assume that the two source Shapefile datasets have the same file name (feature type name) but have different schema and you need to read schema feature for each file with the same FeatureReader.
A possible way is:
- create a custom transformer containg a FeatureReader and publish the Group By parameter to make it a group-based transformer,
- in the main workflow, you can set an initiator's attribute that stores source Shapefile path to the Group By parameter of the custom transformer to read schema separately for each dataset, even if the different datasets have an idential file name.
To publish the Group By parameter in a custom transformer definition, right-click on Transformer Parameters/Group By and select "Create User Parameter", on the Navigator of the canvas tab for defining the custom transformer. See also the screenshot below.
Hi Takashi
Thank you!!!