Skip to main content
Solved

Help with syntax when using Tester transformer

  • November 28, 2023
  • 5 replies
  • 69 views
aaronm2800
Participant
Forum|alt.badge.img

Hi, I need some help setting a basic test with the tester transformer. I'm trying to fail rows from a feature if they match any values in a csv file I've added as a reader. I'm having some issues getting things configured correctly.

 

So something like NOT @Value(field) Contains <Right value>. But I can't seem to connect the right value. Connecting the csv as an input isn't what I want, but I cant figure out the syntax to use in conjunction with '$(SourceDataset_CSV2)'.

 

Sorry for this basic question but I couldn't figure this out looking at the documentation. Thank you.

Best answer by hkingsbury

A tester only works at the feature level, so you can't look at data that isn't on the feature.

 

One approach (the simplest for me to explain and not a particularly efficient solution) is to read your CSV file in as a text file and make sure you're reading the whole file at once (so it comes on one feature)image 

Then pass this to a featuremerger as the supplier and pass your other data to the requestor port. In the feature merger set the join to '1=1'imageComing out of the merged port, you should now have the same number of features coming in your requestor port with an additional attribute containing the entire csv.

 

Now you can pass this into the test with your original contains statement with the right value now being the attribute containing the csv file

This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

5 replies

hkingsbury
Celebrity
Forum|alt.badge.img+65
  • hkingsburyCelebrity
  • Celebrity
  • Best Answer
  • November 28, 2023

A tester only works at the feature level, so you can't look at data that isn't on the feature.

 

One approach (the simplest for me to explain and not a particularly efficient solution) is to read your CSV file in as a text file and make sure you're reading the whole file at once (so it comes on one feature)image 

Then pass this to a featuremerger as the supplier and pass your other data to the requestor port. In the feature merger set the join to '1=1'imageComing out of the merged port, you should now have the same number of features coming in your requestor port with an additional attribute containing the entire csv.

 

Now you can pass this into the test with your original contains statement with the right value now being the attribute containing the csv file

aaronm2800
Participant
Forum|alt.badge.img
  • aaronm2800Participant
  • Author
  • Participant
  • November 28, 2023

A tester only works at the feature level, so you can't look at data that isn't on the feature.

 

One approach (the simplest for me to explain and not a particularly efficient solution) is to read your CSV file in as a text file and make sure you're reading the whole file at once (so it comes on one feature)image 

Then pass this to a featuremerger as the supplier and pass your other data to the requestor port. In the feature merger set the join to '1=1'imageComing out of the merged port, you should now have the same number of features coming in your requestor port with an additional attribute containing the entire csv.

 

Now you can pass this into the test with your original contains statement with the right value now being the attribute containing the csv file

Thank you for the idea. Just to clarify, would this add the entire csv as an attribute column for each row?

hkingsbury
Celebrity
Forum|alt.badge.img+65
  • hkingsburyCelebrity
  • Celebrity
  • November 28, 2023

Thank you for the idea. Just to clarify, would this add the entire csv as an attribute column for each row?

As its being read in as a text file (not as a csv) it treats each row as a new feature (doesn't split it into columns), then, because the reader is specified to read the whole file at once, it actually reads the complete CSV file into one attribute, on one feature. It's then treated as a text field

aaronm2800
Participant
Forum|alt.badge.img
  • aaronm2800Participant
  • Author
  • Participant
  • November 29, 2023

Thank you for the idea. Just to clarify, would this add the entire csv as an attribute column for each row?

Thanks again, it does seem strange to me there isn't a more efficient way to do something equivalent to 'if x in y: <action>'.

hkingsbury
Celebrity
Forum|alt.badge.img+65
  • hkingsburyCelebrity
  • Celebrity
  • November 29, 2023

Thank you for the idea. Just to clarify, would this add the entire csv as an attribute column for each row?

In a way, that is exactly what youre doing (if x in y) FME works at a feature level, so that statement is true for the context of a feature, hence 'loading' the whole csv into one attribute

 

A more 'complex' and efficient approach is to use an inlinequerier which loads your input data into an SQLite database which then allows you to write a SQL statement to do 'database level' joins etc

Community Stats

32,154
Posts
122,047
Replies
39,812
Members

Latest FME

Terms & ConditionsCookie settingsAccessibility statement
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.