Skip to main content
Question

En-masse data translations

  • July 13, 2023
  • 6 replies
  • 25 views

scarter
Contributor
Forum|alt.badge.img+10

 Hello Group,

 

I manage a large directory of geospatial data, mostly in .tab. But times and needs are changing and I’d like to have the data available in other formats such as shp, geojson and gpkg.

 

In an ideal world we’d have a PostGIS database, but that’s not about to happen any time soon. So, for the time being, and probably into the foreseeable, I will continue to maintain flat files.

 

In FME Form, I need a process where I can point a FeatureReader at a folder and, in as few steps as possible, write each one to .shp, for example. I need to maintain the name (done so through Accumulation Mode: Merge Initiator and Result, I believe). So, for example a folder may contain 20 tab files (Zones_A.tab, Zones_B.tab, Zones_C.tab etc. etc.). I would want to create Zones_A.gpkg, Zones_B.gpkg, Zones_C.gpkg, but do so in one go.

 

I know I can read in files from a folder (via ‘Select Multiple folders/Files’ in the FeatureReader) and then publish as a different format to the same location (or different location) by connecting each read-in layer to a Connect Input port in the FeatureWriter. See image.

 

Whilst that’s not too time consuming if it’s a one-off or just a handful of layers, but what if the folder contains 150 .tabs that need translating? What if I want to run this process again on another 50 folders, for example. I certainly would to want to spend all day dragging out connecter lines manually.

 

Is there a more streamlined way of doing all of this?

 

To confirm: in a few steps possible I want read in all the .tabs in one specified folder, and then write out those datasets to a specified folder (or the same one) in a specified data format (e.g.: gpkg). And to do this without having drag out connecter lines to Connect Input ports in the FeatureWriter.

 

ThanksFME130723

6 replies

tomfriedl
Contributor
Forum|alt.badge.img+13
  • Contributor
  • 177 replies
  • July 13, 2023

The input feature have an attribute fme_feature_type. You can use it in the FeatureWriter:

fw


nielsgerrits
VIP
Forum|alt.badge.img+60
  • 2938 replies
  • July 13, 2023

scarter
Contributor
Forum|alt.badge.img+10
  • Author
  • Contributor
  • 125 replies
  • July 13, 2023

The input feature have an attribute fme_feature_type. You can use it in the FeatureWriter:

fw

Thanks for that. So, I take it I have to use an AttributeExposer to expose fme_feature_type. Is that right? But in this example I am reading in 23 datasets in the one FeatureReader. Is there a way of avoiding dragging out 23 connection lines to the AttributeExposer?

tomfriedl
Contributor
Forum|alt.badge.img+13
  • Contributor
  • 177 replies
  • July 13, 2023
Thanks for that. So, I take it I have to use an AttributeExposer to expose fme_feature_type. Is that right? But in this example I am reading in 23 datasets in the one FeatureReader. Is there a way of avoiding dragging out 23 connection lines to the AttributeExposer?

Parameter FeatureReader: Output > Single Output Port. In this case all feature leave through the <generic>-port. it is not necessary to expose fme_feature_type


scarter
Contributor
Forum|alt.badge.img+10
  • Author
  • Contributor
  • 125 replies
  • July 13, 2023
Thanks for that. So, I take it I have to use an AttributeExposer to expose fme_feature_type. Is that right? But in this example I am reading in 23 datasets in the one FeatureReader. Is there a way of avoiding dragging out 23 connection lines to the AttributeExposer?
Thanks for that. I have… · Creator, connected to… · FeatureReader · In FeatureReader I’m pointing to the folder containing the 23 .tab files I want to translate · In FeatureReader I’ve set Output>Output Ports to Single Output Port · FeatureReader is connected to FeatureWriter · In FeatureWriter I have set shp as the format, and specified a destination folder. When I run this I get one shape file produced called _Generic_. But I need 23 x .shp files, each with the same name as the input .tab. Do I need to fanout? Thanks,

scarter
Contributor
Forum|alt.badge.img+10
  • Author
  • Contributor
  • 125 replies
  • July 13, 2023
Thanks for that. So, I take it I have to use an AttributeExposer to expose fme_feature_type. Is that right? But in this example I am reading in 23 datasets in the one FeatureReader. Is there a way of avoiding dragging out 23 connection lines to the AttributeExposer?
Thanks for that. I have… · Creator, connected to… · FeatureReader · In FeatureReader I’m pointing to the folder containing the 23 .tab files I want to translate · In FeatureReader I’ve set Output>Output Ports to Single Output Port · FeatureReader is connected to FeatureWriter via the Generic port · In FeatureWriter I have set shp as the format, and specified a destination folder. When I run this I get one shape file produced called _Generic_. But I need 23 x .shp files, each with the same name as the input .tab. Do I need to fanout? Thanks,