Solved

Allow Attributes Based on Other Reader

pratap
Rank
Badge +2

Hello Everyone,

I have several readers & writers, I want to restrict the attributes of shape file based on excel file (other reader).

Eg: I have shape file with 25 attributes, out of which I may require 10 attributes for translation. As of now I'm using attributekeeper after reader to restrict the attributes which are not required.

For further explanation, whatever the attribute names I pass through the excel file only those attributes have to be passed from shape file.

Note: Excel file contains only attribute headers

Is there a way to perform this???

Thank you

icon

Best answer by takashi 3 December 2015, 05:42

View original

12 replies

takashi
Rank
Userlevel 2
Badge +17

Do you need dynamic schema configuration based on the attribute names list in the Excel file?

pratap
Rank
Badge +2

Hello Takashi,

Thank you for the reply.

Seems to be "Yes" to your question. I have not used dynamic schema until now.

Pratap

pratap
Rank
Badge +2

Hi,

I don't want to change the schema of writer because it has several other attributes and it is constant.

Pratap

takashi
Rank
Userlevel 2
Badge +17

The Schema (Any Format) reader can be used to read schema features from any dataset. A schema feature contains a structured list called "attribute{}", which stores attribute names ("attribute{}.name") and their data type names defined in a feature type of the source dataset.

My idea is:

  1. Add a Schema (Any Format) reader to read schema feature from the source Shape file, and explode the feature on "attribute{}" using the ListExploder.
  2. As well, add another Schema reader for the Excel file, and explode the schema feature.
  3. Merge the exploded features from Excel to the exploded features from Shape using "name" (i.e. attribute name) as the join key.
  4. Re-build "attribute{}" list from the Merged features using the ListBuilder or Aggregator. The re-built "attribute{}" list consists of only attributes coming from the Excel.
  5. Merge the list to the output line features unconditionally.

You can then configure dynamic writer schema with the method described in this article: Destination Schema is Derived from List Attributes

takashi
Rank
Userlevel 2
Badge +17

If you don't want to change writer schema, just configure the schema statically?

pratap
Rank
Badge +2

I have made sample files and tried but unable to get the result as required as well as geometry as well.

Attached are the files with which I have tried. safe.zip

Could you please modify the workspace if I'm wrong

takashi
Rank
Userlevel 2
Badge +17

See this example: schema-schema2none.zip

G
Badge +3

Merge the excell schema with the writer schema?

If writer schema contains attributes from shape wich are to be discarded, you must alsoo remove those.

To do this you need to provide both writer schema (you said it is fixed, so this should be no problem) and the excell schema.

Use in a dynamic set-up.

mark2atsafe
Rank
Userlevel 4
Badge +25

I agree with Takashi, if your schema is static - i.e. the Excel file will never change you know that you only need attributes 2,4,6,7 - then edit the Writer schema to only include those attributes.

Alternatively, I can think of two other solutions for where it is not static (i.e. the Excel file might change):

1) Set the Writer schema to Automatic. Hide all the attributes on the Shape Reader, and connect both it and the Excel to the Writer. The schema will come from the Excel (and not from the Shape because its attributes are hidden) but the attribute values will come from the Shape (which are only hidden, not deleted). This works (I tried it) but it isn't truly dynamic - if the Excel changes you would need to update the Reader schema manually.

2) Set the Writer schema to Dynamic. Add the Excel as a Reader Resource. Set the source of the Writer schema to be the Excel dataset. Now only Shape attributes matching that dataset get written.

Although I too went down the exploded list route at first, I really don't think it is necessary. I'm quite pleased with how the first alternative works!

mark2atsafe
Rank
Userlevel 4
Badge +25

I agree with Takashi, if your schema is static - i.e. the Excel file will never change you know that you only need attributes 2,4,6,7 - then edit the Writer schema to only include those attributes.

Alternatively, I can think of two other solutions for where it is not static (i.e. the Excel file might change):

1) Set the Writer schema to Automatic. Hide all the attributes on the Shape Reader, and connect both it and the Excel to the Writer. The schema will come from the Excel (and not from the Shape because its attributes are hidden) but the attribute values will come from the Shape (which are only hidden, not deleted). This works (I tried it) but it isn't truly dynamic - if the Excel changes you would need to update the Reader schema manually.

2) Set the Writer schema to Dynamic. Add the Excel as a Reader Resource. Set the source of the Writer schema to be the Excel dataset. Now only Shape attributes matching that dataset get written.

Although I too went down the exploded list route at first, I really don't think it is necessary. I'm quite pleased with how the first alternative works!

writeshapeattrsfromexcel.fmwt

Here's a workspace template showing the first alternative method. The main problem I had is in data types (without data in the Excel FME defined them both as char(2) data types).

pratap
Rank
Badge +2

Thank you everyone for the responses.

@takashi, I have disabled the writer and checked the result with inspector but result is not matching as I required.

Seems like I have not posted clearly. I don't want to define writer schema or reader (i.e. point shape file as in this example) schema, those schema's are constant but contents of the excel file will change. As I told earlier, I'm controlling the attributes for translation with attribute keeper transformer. I want to replace this procedure with the help of excel file as it is easy for me. I receive excel file as one of the input i.e. Version 1.0 or Version 1.1. In this example I require ATTR_2,ATTR_5,ATTR_7,ATTR_9 attributes for translation but when I receive another version of excel I may require ATTR_3,ATTR_5,ATTR_6,ATTR_8

In this example the result should be point shape file with 16 points with ATTR_2,ATTR_5,ATTR_7,ATTR_9,ATTR_12,ATTR_13 attributes

i.e.ATTR_12 & ATTR_13 are additional attributes in writers and will be null

Pratap

takashi
Rank
Userlevel 2
Badge +17

The Dynamic Schema mechanism only works for actual writing destination dataset, will not be applied to the Inspector. If you run my workspace example, the resulting shape file only contains these attributes:

ATTR_2, ATTR_5, ATTR_7, ATTR_9

If you need to add ATTR_12 and ATTR_13, you can add them manually in the writer feature type (User Attributes tab).

Doesn't this satisfy your requirement?

Reply


Community Stats

28,670
Posts
109,365
Replies
36,531
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.