• Home
  • Forums
  • FME Form
  • Transformers
  • I need retrospectively run/play back through a dataset by date and apply an attribute based on frequency of records with the same name within 10 days of the current date.
Question

I need retrospectively run/play back through a dataset by date and apply an attribute based on frequency of records with the same name within 10 days of the current date.

T

I have a dataset that has been collated daily over the past year but a crucial attribute hasn't been collected which would have taken the submission date of the record and looked back to see if any matching records by Name over the last 10 days and then if true write a True or false into an attribute.

Is there an easy way run/play back through the dataset by day and calculate the True\\False attribute and then add to the result, I need to consider all dates cumulatively so the first date I would be just records for that day then 2nd date would be the processed records and the records for the next date etc.

It feels like it should be quite a simple thing to do but I can't get my head around it at the moment, any help would be much appreciated.

 

Many Thanks

 

Tom

5 replies

nielsgerrits
Rank
Userlevel 6
Badge +32

Can you provide sample data in Excel so we can see and understand what you need and try to reproduce it in FME?

 

Now I need to create my own set of sample data based on your description which is time consuming and prone to error due to interpretation.

T

@nielsgerrits​ Thanks for the reply, I have attached a sample Excel table.

In the attached table I have applied the Y/N true or false flag for Names A-L based on the logic if there are 5 or more with the same name in the previous 10 days then they will all be flagged with 'Y'.

I have the FME workbench built to do apply the Y/N flag but the piece I can't get to grips with is processing the data as if it was coming in day by day and applying the Y/N flag over time, basically retrospectively simulating and processing the data coming in as it did on a daily basis.

 

Thanks,

 

Tom

1 Attachment
nielsgerrits
Rank
Userlevel 6
Badge +32

@nielsgerrits​ Thanks for the reply, I have attached a sample Excel table.

In the attached table I have applied the Y/N true or false flag for Names A-L based on the logic if there are 5 or more with the same name in the previous 10 days then they will all be flagged with 'Y'.

I have the FME workbench built to do apply the Y/N flag but the piece I can't get to grips with is processing the data as if it was coming in day by day and applying the Y/N flag over time, basically retrospectively simulating and processing the data coming in as it did on a daily basis.

 

Thanks,

 

Tom

I have seen a solution previously where someone created visual timeline using the date field. Draw a point on de X axis. Create an area for a range, 10 units back. Then look how much points fall in the area.

2022-05-03_15h03_202022-05-03_15h02_21

geomancer
Rank
Userlevel 4
Badge +36

"Treat it like geometries", interesting approach!

To do: for name = A the attribute _overlaps = 6 on date = 20210912, so name = A will get a value True in the appropriate attribute for all records of A that exist on this day and the 9 preceding days.

T

I have seen a solution previously where someone created visual timeline using the date field. Draw a point on de X axis. Create an area for a range, 10 units back. Then look how much points fall in the area.

2022-05-03_15h03_202022-05-03_15h02_21

Thank you for the suggestion, definitely an approach I hadn't thought of, I'll test and see if I can apply it to my workflow and provide feedback.

 

Regards,

Tom

Reply


Community Stats

28,669
Posts
109,359
Replies
36,531
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.