Question

Filtering date values with a Workspace Runner

crutledge
Rank
Badge +8

Hi All,

Hoping a fresh set of eyes may be able to identify why the workbench below performs differently when run on its own and when called on using a workspace runner. The intent of the workflow is to read a table of gps points and filter them into 'Runs' between a Start and End time. I have two branches for testing right now. One writes to a spreadsheet and one to csv. I am expecting that all the runs will be written out to the spreadsheet while each run will be written out to their own csv file using the unique RunName. I don't think it is an issue of datetime format. The workspace below runs with test values in the User Parameters but when the Workspace runner reads the table of Start/Stop Runs and feeds the StartTime EndTime and Runname to the workbench only one csv is output while the spreadsheet contains a larger but incomplete dataset of filtered gps points.

So the spreadsheet gets more data but incomplete while only one csv is written out when I am expecting many csv files one per RunName.

Second screencap below shows that there are 411 workspace runs but the results are not as expected. Thoughts? Much appreciated.

 

TimeOnRunClassifierTimeOnRunWorkspaceRunner

3 replies

caracadrian
Rank
Badge +20

So, you set 3 Parameters (StartTime, EndTime and RunName) via attributes in WorkspaceRunner.

Then, you use these parameters to filter data and write two files .xlsx and .csv

Are the datetime parameters set as datetime or as string? Maybe FME passes datetime differently via WorkspaceRunner and not ISO datetime. If I set a Datetime parameter and put it in an attribute via ParameterFetcher and the value comes out as FME datetime not ISO datetime. And it should be to do math (smaller, larger) on it. If you pass an ISO datetime via WorkspaceRunner then the filter won't work.

Is RunName unique when it gets passed on? That may be the cause of a single csv being writen.

How is the Fanout set for your Excel Writer?

Do you actually need to do this via WorkspaceRunner? Can it be made in just a single workspace?

crutledge
Rank
Badge +8

So, you set 3 Parameters (StartTime, EndTime and RunName) via attributes in WorkspaceRunner.

Then, you use these parameters to filter data and write two files .xlsx and .csv

Are the datetime parameters set as datetime or as string? Maybe FME passes datetime differently via WorkspaceRunner and not ISO datetime. If I set a Datetime parameter and put it in an attribute via ParameterFetcher and the value comes out as FME datetime not ISO datetime. And it should be to do math (smaller, larger) on it. If you pass an ISO datetime via WorkspaceRunner then the filter won't work.

Is RunName unique when it gets passed on? That may be the cause of a single csv being writen.

How is the Fanout set for your Excel Writer?

Do you actually need to do this via WorkspaceRunner? Can it be made in just a single workspace?

Hi @caracadrian thanks for the quick reply!

Second set of eyes gives me some things to check and confirm.

Parameters were set as datetime so should be translating readable formats but will test using ParameterFetcher.

Good to know the greater than and less than math should be working so pointing to formats passed from Workspace runner.

RunName to ensure RunName is unique I concatenated the RunName with Timestamp.

Will check what exactly what format is being passed via workspace runner ISO datetime or FME datetime.

No Fanout for the Excel Writer. Simply appends to file after each run.

Don't know if I need a workspace runner but I need to read records from one Spreadsheet/Table with a StartTime, EndTime and RunName and then read a second Spreadsheet/Table that contains a collection of GPS points with TimeStamp and Lat, Lon. So the workflow I envisioned was to Loop through the list of RunNames with Start End time and filter/group the GPS points table to later 'construct' lines from each of the collection of points. Make sense?

Thanks for taking the time to share your thoughts!

Best,

c.

crutledge
Rank
Badge +8

Hi All,

Still stuck on the workspace only wanting to write out the last run from the Workspace runner. Made sure to check the params on the way in and set in a format that should be able to be filtered. I mean it does filter but only for one run.

Any ideas as to why the workspace below would not write out a csv for each call by the workspace runner? Values being passed over seem to be fine. I can post the workspaces....just thought someone might be able to catch something quick from the screen caps. Thanks for your time and ideas.

Best,

c.

image.pngimage

Reply


Community Stats

28,631
Posts
109,175
Replies
36,511
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.