Skip to main content
Solved

copy one column each from 30 files into one

  • Nov 27, 2013 10:08 PM UTC
  • 8 replies
  • 92 views

This should be simple.

 

I have 30 shapefiles of the same points (300) with one column which is the elevation taken from a DEM (ie 30 different DEMs queried with the same point theme). For analysis, I would like all 30 columns to be in one file (e.g. csv, dbf). tried my luck with featuremerger but with no success.

 

Thanks, Uwe

Best answer by takashi

Hi,

 

 

The BulkAttributeRenamer and the FeatureMerger can do that.

 

This is a simplified example, has only 3 input tables.

 

  Assume input shape file names are "name1.shp", "name2.shp",...; each input table has attributes named "ID" and "elevation". 1) Add a SHP Reader to read all shape files with "Single Merged Feature Type" option, expose "fme_feature_type" (holding shp file name) on "Feature Type Properties" dialog box.

 

  2) Use the BulkAttributeRenamer to rename "elevation" to value of "fme_feature_type" (i.e. shp file name) with these settings.   Rename: Selected Attributes   Selected Attributes: elevation   Action: Regular Expression Replace   Test To Find: .+

 

  (dot and plus symbol. It means one or more any characters)   String: fme_feature_type

 

3) Split feature flow into "name1" features and others with the Tester.

 

4) Merge all features with the FeatureMerger.   Requestor to Supplier Join On: "ID" to "ID"   Process Duplicate Suppliers: Yes  <-- important!

 

5) Expose renamed attributes (name1, name2, ...) with the AttributeExposer.

 

Takashi
View original
This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

8 replies

  • Author
  • Nov 27, 2013 10:18 PM UTC
Forgot to mention:

 

Each shape file has a name that should then become the column heading in the new file.

takashi
Celebrity
  • Nov 27, 2013 11:33 PM UTC
Hi,

 

 

Do columns look like this? Source shape files: file name = name1.shp, name2.shp ... name30.shp | elevation |   Expected destination table: name* contains "elevation" from "name*.shp". | name1 |  name2 | name3 | ... | name30 | 

 

 

In the source tables, is there a common attribute which can be used for "Join On" of the FeatureMerger?

 

 

Takashi

  • Author
  • Nov 28, 2013 8:35 AM UTC
Thanks Takashi,

 

excactly.

 

each shape file contains in addition to the elevation attribute  the same point ID field to join on which should make it easy as it is a straight join except that in e.g. ArcMap one needs to do this one at a time and as the task is very likely to come up again and again, a neat FME solution would be great.

 

 

Many thanks, uwe

takashi
Celebrity
  • Best Answer
  • Nov 28, 2013 9:29 AM UTC
Hi,

 

 

The BulkAttributeRenamer and the FeatureMerger can do that.

 

This is a simplified example, has only 3 input tables.

 

  Assume input shape file names are "name1.shp", "name2.shp",...; each input table has attributes named "ID" and "elevation". 1) Add a SHP Reader to read all shape files with "Single Merged Feature Type" option, expose "fme_feature_type" (holding shp file name) on "Feature Type Properties" dialog box.

 

  2) Use the BulkAttributeRenamer to rename "elevation" to value of "fme_feature_type" (i.e. shp file name) with these settings.   Rename: Selected Attributes   Selected Attributes: elevation   Action: Regular Expression Replace   Test To Find: .+

 

  (dot and plus symbol. It means one or more any characters)   String: fme_feature_type

 

3) Split feature flow into "name1" features and others with the Tester.

 

4) Merge all features with the FeatureMerger.   Requestor to Supplier Join On: "ID" to "ID"   Process Duplicate Suppliers: Yes  <-- important!

 

5) Expose renamed attributes (name1, name2, ...) with the AttributeExposer.

 

Takashi

takashi
Celebrity
  • Nov 28, 2013 10:00 AM UTC
Alternatively, the Aggregator (Group By: ID, Keep Input Attributes: Yes) can be used instead of the Tester and FeatureMerger. But the Aggregator will transform input points into multi-point features. If you don't need geometries in the destination dataset, the GeometryRemover can be used before aggregating.   The BulkAttributeRenamer approach is inspired from Ken's suggestion in this thread: Attribute renaming using attribute values http://fmepedia.safe.com/AnswersQuestionDetail?id=906a0000000cnH4AAI

  • Author
  • Nov 28, 2013 10:07 AM UTC
Thanks Takashi,

 

As I am still working on FME 2010 (large organisations are  a bit slow sometimes) there is no BulkAttributeRenamer or even the AttributeExpressionRenamer that is mentioned by Ken in the link in your message :-(

takashi
Celebrity
  • Nov 28, 2013 10:26 AM UTC
Unlucky... well, I would use the PythonCaller with this script. ----- import pyfme def renameAttribute(feature):     value = feature.getAttribute('elevation')     name = feature.getAttribute('fme_feature_type')     feature.setAttribute(name, value) -----   Small company can decide something quickly, but funds are insufficient...

  • Author
  • Nov 28, 2013 8:17 PM UTC
Again, many thanks Takashi!!

 

With the browser at home I can also see the workspace picture which I could not at work. I'll have a go at things the next few days but am sure this will do the trick.

 

Best regards, Uwe