How to filter a csv file based on the last row in FME

Hi!

 

I wonder if using Pointclouds could have a performance-impact positively/negatively here ?

For instance like the screenshot below: You can get the last value in the text-file with PointCloud-operations. (Don't think you can do the actual filtering without changing the component to some Integer though. Would be cool though.).

 

 

Now - this if the case really is the last record in the file.

@daleatsafe , @xiaomengatsafe?

It is true that you can get the last record of a point cloud super fast. And if you ensured you'd get one point cloud per input file (which you do), then this would be an even better way to get the global variables set on the first pass. You still would need to do the second pass to actually set the values of each record based on the last one -- and that would be best done using the traditional CSV reader.

 

 

Note that if we made the Sampler understand our new feature table technology AND if we had a way to get the LAST records from the Sampler, then this becomes as fast using non-pointcloud approaches.

 

 

(In fact, the feature tables are inspired by the speed that FME's point cloud subsystem exhibits).

 

 

Hi, @sobanmughal and @1spatialdave I'm happy to report: as of Build 18187 in the current 2018 Beta, Sampler has a new option to return the "Last N Features". This might be useful for workflow like this one, in the future. Thanks!

Good to hear. Thanks for the enhancement.