Skip to main content
Solved

Multiple CSV with different structure to SHP (dynamic way)

G
Forum|alt.badge.img

Hello

I have 150 CSV files. I would like to convert them to equivalent SHP files. I don't want to do it one by one of course hence I chose dynamic workflow. This should be just a mirror translation (with geometry creation from WKT along the way). But I can't get this to work.

In the writer I tick the Dynamic Properties with Schema Sources set to "*.csv [CSV2]" but this doesn't create any shapefiles complaining: 'Features with no schema defined'

When I untick the Dynamic Properties the shapefiles are created but only with the geometry, no attributes.

Here's the workbench. The CSV filename is the name of the feature. Bunch of attribute renaming and removing. Creating geometries.

Reader:

Writer (I tried geometry defined by schema as well but same result):

Am I missing something?

Thanks!

Best answer by takashi

I suspect that the feature type name has been set to "CSV" (format name).

The CSV reader gives the format name "CSV" as "fme_feature_type" attribute to every feature by default, and the source schema definition will be identified with the "fme_feature_type".

In your writer feature type setting, you intend to identify the source schema definition with "Default from Shapefile name above" i.e. the source file name, but the writer won't do it if the actual source schema definition name (i.e. fme_feature_type) was "CSV".

If you want to use source file names to identify source schema definitions, you have to set "From File Name(s)" to the Feature Type Name(s) parameter when you add the reader to the workspace.

You can then use the "fme_feature_type" as the destination feature type name (Shapefile Name = Schema Definition Name) in the dynamic writer feature type.

View original
Did this help you find an answer to your question?

6 replies

J
Forum|alt.badge.img+2

Hi @gonzikcz

 

What if you set it to "Schema From Schema Feature", does that work?

 

takashi
Influencer
  • takashiInfluencer
  • Best Answer
  • December 9, 2017

I suspect that the feature type name has been set to "CSV" (format name).

The CSV reader gives the format name "CSV" as "fme_feature_type" attribute to every feature by default, and the source schema definition will be identified with the "fme_feature_type".

In your writer feature type setting, you intend to identify the source schema definition with "Default from Shapefile name above" i.e. the source file name, but the writer won't do it if the actual source schema definition name (i.e. fme_feature_type) was "CSV".

If you want to use source file names to identify source schema definitions, you have to set "From File Name(s)" to the Feature Type Name(s) parameter when you add the reader to the workspace.

You can then use the "fme_feature_type" as the destination feature type name (Shapefile Name = Schema Definition Name) in the dynamic writer feature type.

G
Forum|alt.badge.img
  • gonzikcz
  • Author
  • December 12, 2017
takashi wrote:

I suspect that the feature type name has been set to "CSV" (format name).

The CSV reader gives the format name "CSV" as "fme_feature_type" attribute to every feature by default, and the source schema definition will be identified with the "fme_feature_type".

In your writer feature type setting, you intend to identify the source schema definition with "Default from Shapefile name above" i.e. the source file name, but the writer won't do it if the actual source schema definition name (i.e. fme_feature_type) was "CSV".

If you want to use source file names to identify source schema definitions, you have to set "From File Name(s)" to the Feature Type Name(s) parameter when you add the reader to the workspace.

You can then use the "fme_feature_type" as the destination feature type name (Shapefile Name = Schema Definition Name) in the dynamic writer feature type.

Hi @takashi

 

Yes, that fixed it. Thanks for the tip! However, I wonder how is this different from the filename_extractor which returns exactly same table name?

 

Cheers, Jan

 

 

G
Forum|alt.badge.img
  • gonzikcz
  • Author
  • December 12, 2017
jlutherthomas wrote:

Hi @gonzikcz

 

What if you set it to "Schema From Schema Feature", does that work?

 

Hi @jlutherthomas

 

 

I tried this one too but the result was the same. This was my first try of dynamic flow but from the documentation I understood that this option is for different cases.

 

 

Thanks, Jan
takashi
Influencer
  • takashiInfluencer
  • December 13, 2017
takashi wrote:

I suspect that the feature type name has been set to "CSV" (format name).

The CSV reader gives the format name "CSV" as "fme_feature_type" attribute to every feature by default, and the source schema definition will be identified with the "fme_feature_type".

In your writer feature type setting, you intend to identify the source schema definition with "Default from Shapefile name above" i.e. the source file name, but the writer won't do it if the actual source schema definition name (i.e. fme_feature_type) was "CSV".

If you want to use source file names to identify source schema definitions, you have to set "From File Name(s)" to the Feature Type Name(s) parameter when you add the reader to the workspace.

You can then use the "fme_feature_type" as the destination feature type name (Shapefile Name = Schema Definition Name) in the dynamic writer feature type.

The schema definition is identified by its unique name (Schema Definition Name), and the name is equal to the value of "fme_feature_type" if the source schema has been read from a source dataset by a reader. Note that "fme_feature_type" is not always equal to the source file name.

 

In your original workspace, "fme_feature_type" had stored "CSV" (format name) according to the reader parameters setting. It's a case where "fme_feature_type" is not equal to the source file name. Nevertheless you had set the source file name to Schema Definition Name in the writer feature type. It's the reason why the dynamic writer was not able to recognize the schema definition.

 

G
Forum|alt.badge.img
  • gonzikcz
  • Author
  • December 13, 2017
takashi wrote:
The schema definition is identified by its unique name (Schema Definition Name), and the name is equal to the value of "fme_feature_type" if the source schema has been read from a source dataset by a reader. Note that "fme_feature_type" is not always equal to the source file name.

 

In your original workspace, "fme_feature_type" had stored "CSV" (format name) according to the reader parameters setting. It's a case where "fme_feature_type" is not equal to the source file name. Nevertheless you had set the source file name to Schema Definition Name in the writer feature type. It's the reason why the dynamic writer was not able to recognize the schema definition.

 

I see. Thanks for the great explanation!

Reply


Helpful Members This Week

Community Stats

31,300
Posts
118,696
Replies
38,878
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.