Question

Find shapefiles in a folder, merge together and write output

  • 23 April 2020
  • 3 replies
  • 157 views
B

Hi there,

I am fairly new to FME so please go easy!

I'm looking to do the following task, broken down into the following steps:

  1. Search through a folder to locate shapefiles (all line primitives)
  2. Set some criteria to filter which shapefiles I want to keep
  3. Merge & Dissolve the shapefiles
  4. Write the output to a geodatabase

For Step 1 I believe I need to use the 'Directory and File Pathnames' reader, with a *.shp path filter.

For Step 4 I understand I need an Esri Geodatabase writer.

 

For the rest, I am a bit lost...

How do I then inspect what FME has found in Step 1?

What are some useful transformers I could use at each step?

3 replies

M
Badge +2

@bogus I would strongly recommend that you sit in on one of our online training classes (instructor lead or on-demand) that would go over most of what you need here and help you be productive in workbench - all at no cost.

I've attached an example workflow that covers many of the steps you described. If you don't understand a step then searching the KnowledgeBase will probably bring up a short tutorial.

Example workspace (FME 2020): Shape2Geodb.fmwt

B
Badge +3

You are on the right track @bogus .

1. Yes, use a Directory and File Pathnames Reader. This will output all the files or optionally you can set a File Path Filter in the Reader to *.SHP. It doesn't matter as you can also do this Filter within the Workspace using an AttributeFilter (by path_extension), the choice is yours and they will perform virtually the same.

2. The first part of the Workspace, use AttributeFilters, Testers and/or TestFilters to set the criteria for which files you want to keep.

3. Once you have this, the next part of the Workspace is to send the file list Features to a Shape File FeatureReader as the Initiator. This will perform an individual SHP File Read for each File Feature entering the Initiator Port. Whilst the default user input for FeatureReader is to manually choose a single file, FME supports a multi-file feature called "Fan Out" for both Reading and Writing and would recommend reading the help and guides on this. Essentially, if you use a File Path that includes within it a variable Attribute Name like "C:\\My Shape Files\\@Value(SelectedShapeFileName)" then you can get FME to Read or Write to variable file names and handle multi-file situations like this. In your case, since using the Directory and File Pathnames Reader, all you need to set the "Dataset" Parameter to @Value(path_windows) since path_windows is the Attribute with the Full file paths of the SHP File Features

4. I don't believe it will make any different, but on the FeatureReader you should set the Output Parameter to "Single Output Port". Essentially then as FeatureReader iterates through every Shape File, it will Merge all the data together into 1 Output Port and create 1 Feature per Geometry.

5. From there you didn't say whether the GDB is SDE or FGDB, but it is just using a Writer or FeatureWriter to send to either format, noting that the SDE writer is dependent on the bitness (32-bit or 64-bit) or your ArcGIS geoprocessing installation so will only work in the corresponding 32 bit or 64 bit builds of FME WorkBench.

mark2atsafe
Rank
Userlevel 4
Badge +25

You don't actually need to use the Directory/File Pathnames reader. The Shapefile reader will let you read a folder of files. You would just click the drop-down arrow (instead of the browse button) and pick Select Multiple Folders/Files:

The only real reason to use the Directory/Path reader is in selecting which files to keep. Obviously it's better to not read data - rather than read it and discard it immediately - but if the files to read can be selected through this dialog, it would be less complicated than the Directory/Path reader.

From there, you can combine all the features just be sending them all to a Geodatabase writer. If you actually want to merge polygon geometries together then use the Dissolver transformer.

Reply


Community Stats

28,706
Posts
109,467
Replies
36,549
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.