Question

Data filtering

  • 8 February 2023
  • 3 replies
  • 13 views
fmesafe.podpora
Rank
Badge +10

Hello. You don't know which transformer to use. I tried testfilter, but there you can't combine two lines and featuremerger, it's also not quite ideal. I need to filter data in a database with a condition:

- all data with the same uuid

- a point that has the same value of validfrom and validto

validto changes to a value of -1 second down

The result should be validfrom 20220908000000 and validto 20221005235959 in the first line

In the second line validfrom 20221006000000 and validto 99991231235959

Thank you very much.

1 Attachment

3 replies

hkingsbury
Rank
Userlevel 5
Badge +29

I think the key to solving your problem is to get the two features with the same uuid into one feature. A rough order of operations to do this is:

  1. for the first set of input features, prefix "validfrom" and "validto" with "a_"
  2. for the second set of input features, prefix "validfrom" and "validto" with "b_"
  3. using an aggregator, aggregate the two sets of feautres, grouping by the uuid and making sure you merge attributes from all features
  4. you can now use a tester to remove uuids that don't have two "parts" - set the tester to pass features that have both "a_validfrom" and "b_validfrom" values
  5. you can now do an subsequent tests against the "a_" and "b_" values
  6. assuming you want the original features, you can then take the features from the previous step, just keep the uuid and use these with a featuremerger (as the supplier) and pass the original input features into the requestor port, merging on the uuid
Check out the Abley FME Firesides on Youtube: https://www.youtube.com/@ableyltd
fmesafe.podpora
Rank
Badge +10

Hi, that could be the solution, thank you very much for the detailed description, I will try it tomorrow and let you know.

fmesafe.podpora
Rank
Badge +10

I think the key to solving your problem is to get the two features with the same uuid into one feature. A rough order of operations to do this is:

  1. for the first set of input features, prefix "validfrom" and "validto" with "a_"
  2. for the second set of input features, prefix "validfrom" and "validto" with "b_"
  3. using an aggregator, aggregate the two sets of feautres, grouping by the uuid and making sure you merge attributes from all features
  4. you can now use a tester to remove uuids that don't have two "parts" - set the tester to pass features that have both "a_validfrom" and "b_validfrom" values
  5. you can now do an subsequent tests against the "a_" and "b_" values
  6. assuming you want the original features, you can then take the features from the previous step, just keep the uuid and use these with a featuremerger (as the supplier) and pass the original input features into the requestor port, merging on the uuid

Hi,

Sorry for the late reply, but I was trying to test your tutorial today. I've tried different aggregator settings but can't seem to come up with a solution. I must be doing something wrong somewhere. I am attaching the WS test. Thank you

Reply


Community Stats

28,729
Posts
109,527
Replies
36,565
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.