Dear,
I have a folder with 2208 xml files, approx. 30 GB. I read these files using a File Path Reader followed by an XML reader. From these files I select some features and save them locally. However, with my 16 GB RAM I run into memory issues. I already reduced the amount of data as much as possible within my workbench but still its not enough.
- Is there a method I read first 10 xml files – process them – save the outcome and go to the next 10 files, like a loop?
- My second question is whether it will safe memory in case I use a fanout expression or just write one final dataset with the results.
My version is: FME 2023.2.1