Skip to main content
Question

How to go through large set of csv files

  • May 8, 2023
  • 3 replies
  • 43 views

wrijs
Contributor
Forum|alt.badge.img+5
  • Contributor
  • 11 replies

I have a model that has to select rows(points) out of a large collection of large csv files. To encrease performance I want to go throught them one by one. What is the best way to do this?

3 replies

dustin
Influencer
Forum|alt.badge.img+31
  • Influencer
  • 627 replies
  • May 8, 2023

You can select multiple files to read at one time in the dropdown of the reader/parameter.image


wrijs
Contributor
Forum|alt.badge.img+5
  • Author
  • Contributor
  • 11 replies
  • May 23, 2023

Thanks for your replay! this does work. But I have found that workspacerunner also works to go through them one by one

 


dustin
Influencer
Forum|alt.badge.img+31
  • Influencer
  • 627 replies
  • May 23, 2023

Thanks for your replay! this does work. But I have found that workspacerunner also works to go through them one by one

 

WorkspaceRunner does use a little more overhead, as it is opening a separate process for each input. But as long as it's working for you. That's the beauty of FME...multiple ways to arrive at the destination.👍