Skip to main content
Solved

Help with syntax when using Tester transformer

aaronm2800
Participant
Forum|alt.badge.img

Hi, I need some help setting a basic test with the tester transformer. I'm trying to fail rows from a feature if they match any values in a csv file I've added as a reader. I'm having some issues getting things configured correctly.

 

So something like NOT @Value(field) Contains <Right value>. But I can't seem to connect the right value. Connecting the csv as an input isn't what I want, but I cant figure out the syntax to use in conjunction with '$(SourceDataset_CSV2)'.

 

Sorry for this basic question but I couldn't figure this out looking at the documentation. Thank you.

Best answer by hkingsbury

A tester only works at the feature level, so you can't look at data that isn't on the feature.

 

One approach (the simplest for me to explain and not a particularly efficient solution) is to read your CSV file in as a text file and make sure you're reading the whole file at once (so it comes on one feature)image 

Then pass this to a featuremerger as the supplier and pass your other data to the requestor port. In the feature merger set the join to '1=1'imageComing out of the merged port, you should now have the same number of features coming in your requestor port with an additional attribute containing the entire csv.

 

Now you can pass this into the test with your original contains statement with the right value now being the attribute containing the csv file

View original
Did this help you find an answer to your question?

5 replies

hkingsbury
Celebrity
Forum|alt.badge.img+51
  • hkingsburyCelebrity
  • Celebrity
  • Best Answer
  • November 28, 2023

A tester only works at the feature level, so you can't look at data that isn't on the feature.

 

One approach (the simplest for me to explain and not a particularly efficient solution) is to read your CSV file in as a text file and make sure you're reading the whole file at once (so it comes on one feature)image 

Then pass this to a featuremerger as the supplier and pass your other data to the requestor port. In the feature merger set the join to '1=1'imageComing out of the merged port, you should now have the same number of features coming in your requestor port with an additional attribute containing the entire csv.

 

Now you can pass this into the test with your original contains statement with the right value now being the attribute containing the csv file

aaronm2800
Participant
Forum|alt.badge.img
  • aaronm2800Participant
  • Author
  • Participant
  • November 28, 2023
hkingsbury wrote:

A tester only works at the feature level, so you can't look at data that isn't on the feature.

 

One approach (the simplest for me to explain and not a particularly efficient solution) is to read your CSV file in as a text file and make sure you're reading the whole file at once (so it comes on one feature)image 

Then pass this to a featuremerger as the supplier and pass your other data to the requestor port. In the feature merger set the join to '1=1'imageComing out of the merged port, you should now have the same number of features coming in your requestor port with an additional attribute containing the entire csv.

 

Now you can pass this into the test with your original contains statement with the right value now being the attribute containing the csv file

Thank you for the idea. Just to clarify, would this add the entire csv as an attribute column for each row?

hkingsbury
Celebrity
Forum|alt.badge.img+51
  • hkingsburyCelebrity
  • Celebrity
  • November 28, 2023
aaronm2800 wrote:

Thank you for the idea. Just to clarify, would this add the entire csv as an attribute column for each row?

As its being read in as a text file (not as a csv) it treats each row as a new feature (doesn't split it into columns), then, because the reader is specified to read the whole file at once, it actually reads the complete CSV file into one attribute, on one feature. It's then treated as a text field

aaronm2800
Participant
Forum|alt.badge.img
  • aaronm2800Participant
  • Author
  • Participant
  • November 29, 2023
aaronm2800 wrote:

Thank you for the idea. Just to clarify, would this add the entire csv as an attribute column for each row?

Thanks again, it does seem strange to me there isn't a more efficient way to do something equivalent to 'if x in y: <action>'.

hkingsbury
Celebrity
Forum|alt.badge.img+51
  • hkingsburyCelebrity
  • Celebrity
  • November 29, 2023
aaronm2800 wrote:

Thank you for the idea. Just to clarify, would this add the entire csv as an attribute column for each row?

In a way, that is exactly what youre doing (if x in y) FME works at a feature level, so that statement is true for the context of a feature, hence 'loading' the whole csv into one attribute

 

A more 'complex' and efficient approach is to use an inlinequerier which loads your input data into an SQLite database which then allows you to write a SQL statement to do 'database level' joins etc

Reply


Helpful Members This Week

Community Stats

30,942
Posts
117,252
Replies
38,742
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.