Skip to main content
Solved

FeatureReader schema update, dynamically, automatically, csv columns

paetzold_s
Contributor
Forum|alt.badge.img+4

I have the following problem:

 

I have multiple csv files with multiple columns. I want to dynamically change which columns to read.

I have a user parameter to define which columns to read.

 

Even though the " Attributes to Expose" Parameter of my FeatureReader gets updated correctly the Reader dosn't read the correct columns.

Only after opening the properties of the Reader and closing it again by clicking "Ok" the Read reads the correct columns.

 

I'm using FME Desktop 2020.2.3.0 on Windows 10

 

Maybe someone can tell me if this is a bug or if I'm just overlooking something.

 

Many thanks in advance !

Best answer by danminneyatsaf

@paetzold_s​ 

After some discussion with our developers, it looks like this is expected behavior with the FeatureReader. Attributes to expose only affects the authoring environment (the GUI you're editing the transformers in) and doesn't change at runtime. To change which attributes are exposed, you'll need to reopen the FeatureReader and click ok after changing the user parameter's value. As a result we've closed the ticket I shared above.

 

Let me know if you have any follow up questions or clarification.

Dan

View original
Did this help you find an answer to your question?

8 replies

danminneyatsaf
Safer
Forum|alt.badge.img+12

Hey @paetzold_s​ this looks like a bug to me. I've reproduce the issue and have filed an issue to get this resolved. For your reference, the issue number is FMEFORM-29210. We'll make sure to update you here when the issue has been resolved.

paetzold_s
Contributor
Forum|alt.badge.img+4
  • paetzold_sContributor
  • Author
  • Contributor
  • August 11, 2023

Wow, thank you for the quick response. I have found a similar problem with the BulkAttributeRenamer. I'll post it in a new question/thread.

danminneyatsaf
Safer
Forum|alt.badge.img+12
paetzold_s wrote:

Wow, thank you for the quick response. I have found a similar problem with the BulkAttributeRenamer. I'll post it in a new question/thread.

No problem! If you're able to tag me in the question that'd be great. I can take a look and see if the two issues are related.

paetzold_s
Contributor
Forum|alt.badge.img+4
  • paetzold_sContributor
  • Author
  • Contributor
  • August 14, 2023
danminneyatsaf wrote:

No problem! If you're able to tag me in the question that'd be great. I can take a look and see if the two issues are related.

I wasn't sure how to tag/dm you (new to the Forum) so I'll post the link to the second question here:

https://community.safe.com/s/question/0D5Dm00000na4VYKAY/bulkattributerenamer-dynamically-rename-attributes

 

geomancer
Evangelist
Forum|alt.badge.img+48
  • geomancerEvangelist
  • Evangelist
  • August 14, 2023

I notice that you have Feature Cashing active. Does the problem also occur when Feature Caching is off?

paetzold_s
Contributor
Forum|alt.badge.img+4
  • paetzold_sContributor
  • Author
  • Contributor
  • August 15, 2023
geomancer wrote:

I notice that you have Feature Cashing active. Does the problem also occur when Feature Caching is off?

Yes this also happens if Feature Cashing is turned off (tested by writing the output to a csv file).

danminneyatsaf
Safer
Forum|alt.badge.img+12

@paetzold_s​ 

After some discussion with our developers, it looks like this is expected behavior with the FeatureReader. Attributes to expose only affects the authoring environment (the GUI you're editing the transformers in) and doesn't change at runtime. To change which attributes are exposed, you'll need to reopen the FeatureReader and click ok after changing the user parameter's value. As a result we've closed the ticket I shared above.

 

Let me know if you have any follow up questions or clarification.

Dan

paetzold_s
Contributor
Forum|alt.badge.img+4
  • paetzold_sContributor
  • Author
  • Contributor
  • August 16, 2023
danminneyatsaf wrote:

@paetzold_s​ 

After some discussion with our developers, it looks like this is expected behavior with the FeatureReader. Attributes to expose only affects the authoring environment (the GUI you're editing the transformers in) and doesn't change at runtime. To change which attributes are exposed, you'll need to reopen the FeatureReader and click ok after changing the user parameter's value. As a result we've closed the ticket I shared above.

 

Let me know if you have any follow up questions or clarification.

Dan

Thank you for the clarification. This behavior is a bit unintuitve.

If this is the expected behavior how can I model this functionality? I have csv files which don't have a common structure. So at the start of the workflow I prompt the user to set the column numbers for the relevant columns. Then I rename these columns and do further processing.

 

I have 2 potential workarounds:

1) - Set the columns to be exposed by the FeatureReader to e.g. "col0 col1 ... col999"

- rename the relevant columns in an AttributeManager

- filter out all other columns with an AttributeKeeper

 

2) - open the csv file as a txt file

- insert a new header at the top e.g. ;;;relevant_Col_1;;relevant_Col_2;;;;;relevant_Col_3

- expose the cols "relevant_Col_1", "relevant_Col_2" and "relevant_Col_3"

 

both solutions seem a bit inelegant. is there a better way I'm missing?

Reply


Helpful Members This Week

Community Stats

31,223
Posts
118,401
Replies
38,824
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.