Question

Merging gdb feature classes based on similar field names

  • 23 November 2017
  • 8 replies
  • 7 views
B
Badge

Hi,

 

 

I have about 50 feature classes in a geodatabase. In the feature classes, some of the field names are the same but a lot are different.

For example: in one feature class field names are – Sample_ID, Easting_NAD83_UTMz19, Northing_NAD83_UTMz19, Au_INAA_ppb, Ag_INAA_ppm, Fe_ICPMS_ppm, Fe_INAA_ppm, Ca_ICPOES_ppm.

In another feature class, field names are – Sample_ID, Easting_NAD83_UTMz19, Northing_NAD83_UTMz19, Ag_INAA_ppm, Au_ICPMS_ppb, Fe_INAA_ppm.

I want to create a new feature class called “INAA_merged” where every field that contains “_INAA_” will be in it as well as the Sample_ID, Easting/Northing fields. I’d like all the other fields containing for ex: “_ICPMS_” and “_ICPOES_” to be deleted.

 

 

So basically I want to merge all the fields together in one feature class that have the same name and delete all the others.

 

 

Is this possible? Is there a way to set up conditions for merging – ie to keep certain fields if they contain certain characters and to delete others?

8 replies

jdh
Rank
Badge +22

You should have a a look at the BulkAttributeRemover and possibly the BulkAttributeRenamer.

takashi
Rank
Userlevel 2
Badge +17

Hi @badams, in the case where you configure the User Attributes statically in the destination writer feature type, it's not essential to delete unnecessary attributes in the workflow. Keep only required attributes in the User Attributes definition. e.g.

B
Badge

Hi @badams, in the case where you configure the User Attributes statically in the destination writer feature type, it's not essential to delete unnecessary attributes in the workflow. Keep only required attributes in the User Attributes definition. e.g.

Hi @takashi I'm revisiting this after a few months. Thanks for the response. I'm curious about how you set up the whole workflow? I'm pretty new to FME and this sounds like it might be the perfect solution!

 

takashi
Rank
Userlevel 2
Badge +17

Hi @badams, in the case where you configure the User Attributes statically in the destination writer feature type, it's not essential to delete unnecessary attributes in the workflow. Keep only required attributes in the User Attributes definition. e.g.

My intention is very simple. Just connect the two reader feature types to a single writer feature type.

 

 

 

B
Badge
My intention is very simple. Just connect the two reader feature types to a single writer feature type.

 

 

 

Thanks @takashi, but how did you make the triangles yellow?

 

takashi
Rank
Userlevel 2
Badge +17
My intention is very simple. Just connect the two reader feature types to a single writer feature type.

 

 

 

The color of triangles (arrows) are determined by Workbench automatically. Yellow indicates that the attribute in the source is not connected to any attribute in the destination feature type. In other words, the attributes with yellow arrows won't be written into the destination dataset.

 

I've just removed those unnecessary attributes from the destination feature type configuration (User Attributes tab).

 

 

B
Badge
The color of triangles (arrows) are determined by Workbench automatically. Yellow indicates that the attribute in the source is not connected to any attribute in the destination feature type. In other words, the attributes with yellow arrows won't be written into the destination dataset.

 

I've just removed those unnecessary attributes from the destination feature type configuration (User Attributes tab).

 

 

Okay great, thanks @takashi it's working with the way you have it above.

 

 

I'm wondering if there's a way to make this a bit more automated? I might have up to 50 FCs with 50-100 fields each. It gets a little time consuming to have to deselect all the unwanted fields. Do you think there's a way to set up some sort of IF statement? Something like - IF fieldname contains "INAA" write to "_Merged" FC, ELSE delete?

 

rylanatsafe
Rank
Badge +11

Inspired by a previous posting by @takashi ... this workspace template (created in FME 2017.1.2.1) will use the PythonCaller to concatenate all the attribute names from each feature into a new attribute "_attrNames". You could then use the TestFilter (with Regex!) to check for specific strings in that attribute and route those features appropriately.

Cons: If you want one feature to go multiple routes, it will require parallel TestFilters (instead of sequenced)...

This also does not remove any attributes as you have requested, but hopefully this helps the dynamic diversion of the feature flow..! As @jdh suggested, the BulkAttributeRemover could potentially help for that.

filter-on-attribute-name.fmwt

Edit: -er, you might investigate using the BulkAttributeKeeper from FME Hub to keep all attributes that contain desired strings -- attached updated template example.

 

filter-on-attribute-name-revised.fmwt

Reply


Community Stats

28,670
Posts
109,366
Replies
36,531
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.