I have 1.6TB of .tif files.
Each file (11,500 of them) are named numerically and stored in individual .zip files.
I need to convert all of the .tif files to JPEG2000.
So far I ran the tif to Jpeg2000 reader/writer through the batch Deploy on small test runs of 10 files and it worked just fine. But when I run the full meal deal FME freezes.
Please Advise
If you handle this in one workspace, all the .tif files will be loaded in the software, eating your memory. Since you can process these files individually there is no need to load them all at once.
You could try staging this in two workspaces:
- 1st workspace reads directory (Directory and File Pathnames Reader) and starts 2nd workspace via a workspacerunner
- 2nd workspace does the conversion per file.
All you need to do is to add a Published Parameter for the filepath in your original workspace and select this Published Parameter as Dataset. If this works create another workspace to get the filepath of all zips and send these to a workspacerunner.
1st workspace:
2nd workspace:
Read file from published parameter:
Example can be downloaded here: workspacerunnerexample.zip