Renaming Attributes

Hi Ian,

 

 

I think the SchemaMapper is relatively simple way. It's troublesome to edit mapping tables, but you can maintain easily the workspace after once creating those tables. If you have to avoid editing tables, a little complicated workflow will be necessary.

 

  Assume that names and contents of CSV files are followings; every table has 3 columns, doesn't have header (field names row); a header file name consists of some parts with hyphen separated, the first part should be an associated feature type name.   "data.csv" (contains every feature attribute values) TypeA, 1, abc TypeA, 2, def TypeB, 3, ghi TypeB, 4, jkl   "TypeA-Header.csv" (contains attribute names of TypeA feature) TypeName, A1, A2   "TypeB-Header.csv" (contains attribute names of TypeB feature) TypeName, B1, B2   ----- 1. Read and transform "data.csv" features. 1) Put a CSV Reader(1) to read "data.csv"; expose "csv_line_number". Attribute names will be col0, col1, col2.   2) Add a ListPopulator(1) to create a list attribute containing col0, col1, co2 values. Source Attribute Prefix: col List Name: _attr   3) Add a BulkAttributeRenamer(1) to change the simple list to a complex list (_attr{i} --> _attr{i}._value). Rename: Selected Attributes Selected Attributes: _attr{0} _attr{1} _attr{2} Action: Add String Suffix String: ._value   4) Add an AttributeExposer(1) to expose the complex list name. Attributes to Expose: _attr{}._value   2. Read and transform "TypeA-Header.csv" and "TypeB-Header.csv" features. 1) Put a CSV Reader(2) to read those tables; expose "fme_feature_type". Attribute names will be col0, col1, col2.   2) Add an AttributeSplitter to get associated feature type name form "fme_feature_type" (i.e. source file name). Attribute to Split: fme_feature_type Deleimiter or Format String: - List Name: _list   3) Add a ListPopulator(2). Source Attribute Prefix: col List Name: _attr   4) Add a BulkAttributeRenamer(2) (_attr{i} --> _attr{i}._name). Rename: Selected Attributes Selected Attributes: _attr{0} _attr{1} _attr{2} Action: Add String Suffix String: ._name   5) Add an AttributeExposer(2). Attributes to Expose: _attr{}._name   3. Add a FeatureMerger. AttriubteExposer(1) OUTPUT --> REQUESTOR AttributeExposer(2) OUTPUT --> SUPPLIER Join On *use feature type name as key   Requestor: _attr{0}._value   Supplier: _list{0}   4. Add a ListExploder to create features each of which has a pair of "_name" and "_value". List Name: _attr{} Attributes to Keep: List Elements and Original Attributes   5. Add a BulkAttributeRenamer(3) to replace "_value" (attribute name) with an attribute value which is held by "_name". Rename: Selected Attributes Selected Attributes: _value Action: Regular Expression Replace Text to Find: .* String: _name (Set To Attribute Value)   6. Add an Aggregator to re-construct "data" features. Group By: csv_line_number Keep Input Attriubtes: Yes -----   The point is that the BulkAttributeRenamer can be used to replace attribute name with an attribute value. All the processes before the BulkAttributeRenamer(3) are for performing it.   It's possible but the workflow might be a little cluttered. Maybe Python can be used to make the workflow more concise and efficient.

 

  Takashi

These parameter settings for the BulkAttributeRenamer(1) and (2) are also possible, so that the following AttributeExposers can be removed.

 

  Rename: All Attributes Action: Regular Expression Replace Text To Find: } String: }._value   *for the BulkAttributeRenamer(1) String: }._name    *for the BulkAttributeRenamer(2)   In this case, the AttributeSplitter should be moved after the BulkAttributeRenamer(2).

Yes my first choice would also be the SchemaMapper, initially some work needs to be done, but it pays off later on since it makes the schema mapping flexible to maintain.

Yeees! If I got a job like this, I would use the SchemaMapper, and create common tables for both the SchemaMapper and Dynamic Schema :-)

Thanks for all of that.

 

 

Well I did use SchemaMapper but it didn't appear to work. Attributes on the way out along the "Mapped" connecter were all still called col0, col1, etc. Have I missed some important setting somewhere?

Here's the CSV I provided to SchemaMapper

 

 

OldName,NewName

 

col0,RECORD_IDENTIFIER

 

col1,CUSTODIAN_NAME

 

col2,LOCAL_CUSTODIAN_NAME

 

col3,PROCESS_DATE

 

col4,VOLUME_NUMBER

 

col5,ENTRY_DATE

 

col6,TIME_STAMP

 

col7,VERSION

 

col8,FILE_TYPE

 

 

but the output attributes still had their old names

Although the SchemaMapper (Action: Map Attributes) renames attributes internally, new attribute names will not exposed on Workbench interface. Because the new attribute names are configured dynamically at run-time based on the external table but attribute names shown on the interface are configured statically at creating workspace.   Such a situation is similar to a case of creating new attributes using Python / Tcl script. The PythonCaller and the TclCaller have "Attributes to Expose" parameter for exposing new attribute names, but the SchemaMapper doesn't have it.   In order to check renaming result,  you can use the Logger transformer. If you need to refer new attribute names from other transformers, consider using the AttributeExposer to expose explicitly them.

Hi All,

 

 

Yep Itay made the best suggestion in the end. I discovered that AttributeRenamer has an [Import ...] button which lets you import a (CSV) file where you have columns for OldName and NewName. You need to choose the option "Attribute Values Containing Names" and you must also have a column header for the Default value.

 

 

A note for Safe here: might be nice if this Import behaviour looked for columns called OldName, NewName and Default and offered them if it found them to save a bit of faff.