We are uploading large zip files containing a folder structure like this, but much more folders and files in it.
Folder1
- file1
- file 2
Folder 2:
-file3
- file4
The upload is done with the Data Upload Service and we are using the parameter opt_extractarchive to automatically unzip all files.
In a next step, a FME service will go through the unzipped file structure on the server and process these files. We have realized that FME does not process part of the uploaded files, there is always a list of files which are not processed. But with every upload, the files not processed are changing, sometimes the first 30 files are processed well and the next 25 are missing, sometimes the first 35 files are ok...
Could it be that the unzip happens after the upload, and when we start the processing job, maybe not all files are unzipped yet?
We have this issue only with large zip folder structures (they contain a lot of pdf files and images in it).