Skip to main content
Solved

Multiple CSV with different structure to SHP (dynamic way)

  • December 8, 2017
  • 6 replies
  • 27 views
G
Forum|alt.badge.img

Hello

I have 150 CSV files. I would like to convert them to equivalent SHP files. I don't want to do it one by one of course hence I chose dynamic workflow. This should be just a mirror translation (with geometry creation from WKT along the way). But I can't get this to work.

In the writer I tick the Dynamic Properties with Schema Sources set to "*.csv [CSV2]" but this doesn't create any shapefiles complaining: 'Features with no schema defined'

When I untick the Dynamic Properties the shapefiles are created but only with the geometry, no attributes.

Here's the workbench. The CSV filename is the name of the feature. Bunch of attribute renaming and removing. Creating geometries.

Reader:

Writer (I tried geometry defined by schema as well but same result):

Am I missing something?

Thanks!

Best answer by takashi

I suspect that the feature type name has been set to "CSV" (format name).

The CSV reader gives the format name "CSV" as "fme_feature_type" attribute to every feature by default, and the source schema definition will be identified with the "fme_feature_type".

In your writer feature type setting, you intend to identify the source schema definition with "Default from Shapefile name above" i.e. the source file name, but the writer won't do it if the actual source schema definition name (i.e. fme_feature_type) was "CSV".

If you want to use source file names to identify source schema definitions, you have to set "From File Name(s)" to the Feature Type Name(s) parameter when you add the reader to the workspace.

You can then use the "fme_feature_type" as the destination feature type name (Shapefile Name = Schema Definition Name) in the dynamic writer feature type.

This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

6 replies

J
Forum|alt.badge.img+2

Hi @gonzikcz

 

What if you set it to "Schema From Schema Feature", does that work?

 

takashi
Celebrity
  • takashiCelebrity
  • Best Answer
  • December 9, 2017

I suspect that the feature type name has been set to "CSV" (format name).

The CSV reader gives the format name "CSV" as "fme_feature_type" attribute to every feature by default, and the source schema definition will be identified with the "fme_feature_type".

In your writer feature type setting, you intend to identify the source schema definition with "Default from Shapefile name above" i.e. the source file name, but the writer won't do it if the actual source schema definition name (i.e. fme_feature_type) was "CSV".

If you want to use source file names to identify source schema definitions, you have to set "From File Name(s)" to the Feature Type Name(s) parameter when you add the reader to the workspace.

You can then use the "fme_feature_type" as the destination feature type name (Shapefile Name = Schema Definition Name) in the dynamic writer feature type.

Why not inspect features with Visual/Data Preview and Feature/Record Information before writing them into a destination dataset?
G
Forum|alt.badge.img
  • gonzikcz
  • Author
  • December 12, 2017

I suspect that the feature type name has been set to "CSV" (format name).

The CSV reader gives the format name "CSV" as "fme_feature_type" attribute to every feature by default, and the source schema definition will be identified with the "fme_feature_type".

In your writer feature type setting, you intend to identify the source schema definition with "Default from Shapefile name above" i.e. the source file name, but the writer won't do it if the actual source schema definition name (i.e. fme_feature_type) was "CSV".

If you want to use source file names to identify source schema definitions, you have to set "From File Name(s)" to the Feature Type Name(s) parameter when you add the reader to the workspace.

You can then use the "fme_feature_type" as the destination feature type name (Shapefile Name = Schema Definition Name) in the dynamic writer feature type.

Hi @takashi

 

Yes, that fixed it. Thanks for the tip! However, I wonder how is this different from the filename_extractor which returns exactly same table name?

 

Cheers, Jan

 

 

G
Forum|alt.badge.img
  • gonzikcz
  • Author
  • December 12, 2017

Hi @gonzikcz

 

What if you set it to "Schema From Schema Feature", does that work?

 

Hi @jlutherthomas

 

 

I tried this one too but the result was the same. This was my first try of dynamic flow but from the documentation I understood that this option is for different cases.

 

 

Thanks, Jan
takashi
Celebrity
  • takashiCelebrity
  • December 13, 2017

I suspect that the feature type name has been set to "CSV" (format name).

The CSV reader gives the format name "CSV" as "fme_feature_type" attribute to every feature by default, and the source schema definition will be identified with the "fme_feature_type".

In your writer feature type setting, you intend to identify the source schema definition with "Default from Shapefile name above" i.e. the source file name, but the writer won't do it if the actual source schema definition name (i.e. fme_feature_type) was "CSV".

If you want to use source file names to identify source schema definitions, you have to set "From File Name(s)" to the Feature Type Name(s) parameter when you add the reader to the workspace.

You can then use the "fme_feature_type" as the destination feature type name (Shapefile Name = Schema Definition Name) in the dynamic writer feature type.

The schema definition is identified by its unique name (Schema Definition Name), and the name is equal to the value of "fme_feature_type" if the source schema has been read from a source dataset by a reader. Note that "fme_feature_type" is not always equal to the source file name.

 

In your original workspace, "fme_feature_type" had stored "CSV" (format name) according to the reader parameters setting. It's a case where "fme_feature_type" is not equal to the source file name. Nevertheless you had set the source file name to Schema Definition Name in the writer feature type. It's the reason why the dynamic writer was not able to recognize the schema definition.

 

Why not inspect features with Visual/Data Preview and Feature/Record Information before writing them into a destination dataset?
G
Forum|alt.badge.img
  • gonzikcz
  • Author
  • December 13, 2017
The schema definition is identified by its unique name (Schema Definition Name), and the name is equal to the value of "fme_feature_type" if the source schema has been read from a source dataset by a reader. Note that "fme_feature_type" is not always equal to the source file name.

 

In your original workspace, "fme_feature_type" had stored "CSV" (format name) according to the reader parameters setting. It's a case where "fme_feature_type" is not equal to the source file name. Nevertheless you had set the source file name to Schema Definition Name in the writer feature type. It's the reason why the dynamic writer was not able to recognize the schema definition.

 

I see. Thanks for the great explanation!

Community Stats

32,154
Posts
122,048
Replies
39,812
Members

Latest FME

Terms & ConditionsCookie settingsAccessibility statement
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.