Skip to main content

Hi,

I upgraded from 2018.1.2.1 to 2020.0.0.1 on windows 10 machine.

I have a workspace that reads feature classes from the SDE, inline queries, attribute keeper, FeatureWriter (1 per FGDB, SHP, DWG, GEO, CSV, KMZ), before file copy inserts a text disclaimer into the zipped folder. Previously this process worked flawlessly (years), now the featurewriter deletes the previous zipped folder before the FileCopy writes/inserts the disclaimer. If i remove the disclaimer the featurewriter deletes the previous zipped folder, if rerun with nothing in the workspace, it correctly writes out.

I get the warning below when it deletes everything when overwriting.

MULTI_WRITER: Skipped adding of some files to zip file '\\\\ncservfile\\database\\xx\\xx\\xx\\Recreation.zip' because they already exist

Below is a sample of the workspace.

 

Any help would be greatly appreciated!

@dmerrick I could not reproduce your result. Unfortunately, without your sample data I could not run your workspace so (using FME 2019.2) I added a new FeatureWriter and File Copy. The resulting output.zip seems to contain the File Geodb and additional text file. Example workspace (2019.2): featurewriterfilecopy.fmwt


@dmerrick I could not reproduce your result. Unfortunately, without your sample data I could not run your workspace so (using FME 2019.2) I added a new FeatureWriter and File Copy. The resulting output.zip seems to contain the File Geodb and additional text file. Example workspace (2019.2): featurewriterfilecopy.fmwt

@markatsafe, I ran your workspace within 2020.0.0.1 after removing the file copy transformer with the same same problem. The 2nd running of the workspace removes the original zipped output. Attached are the 2 log files.logs.zip

 

I also get a "there is a lock" when clicking on the "Open Containing Folder" icon on the feature writer transformer.

@markatsafe, I ran your workspace within 2020.0.0.1 after removing the file copy transformer with the same same problem. The 2nd running of the workspace removes the original zipped output. Attached are the 2 log files.logs.zip

 

I also get a "there is a lock" when clicking on the "Open Containing Folder" icon on the feature writer transformer.

@dmerrick Did you save the workspace that's in the template before running it? Sometimes the local paths are not set correctly if you just run the workspace directly out of the template file.


@dmerrick Did you save the workspace that's in the template before running it? Sometimes the local paths are not set correctly if you just run the workspace directly out of the template file.

saved and set the write out location locally, same issue persists.

I also got my coworker to try the same process on his computer, and he got the same results with the 2nd running deleting the zipped output entirely.


@dmerrick I have been able to reproduce the issue. It seems to be intermittent and i couldn't determine the cause.

  • Try setting the File Copy parameter Overwrite existing File = No
  • Possibly try the Decelerator to ensure your target file is fully zipped before copying the disclaimer file. Perhaps there is osme latency across your network drives.

This workspace seems to work consistently: Simplefeaturewriterfilecopy.fmwt


@dmerrick I have been able to reproduce the issue. It seems to be intermittent and i couldn't determine the cause.

  • Try setting the File Copy parameter Overwrite existing File = No
  • Possibly try the Decelerator to ensure your target file is fully zipped before copying the disclaimer file. Perhaps there is osme latency across your network drives.

This workspace seems to work consistently: Simplefeaturewriterfilecopy.fmwt

@markatsafe, i have been testing this both locally and on the network, testing your Simplefeaturewriterfilecopy.fmwt still produces the same issue in FME2020.

As a workaround I will look into deleting the zipped files (all 91 files in 13 folders) before running the workspace to write them out.


@dmerrick I have been able to reproduce the issue. It seems to be intermittent and i couldn't determine the cause.

  • Try setting the File Copy parameter Overwrite existing File = No
  • Possibly try the Decelerator to ensure your target file is fully zipped before copying the disclaimer file. Perhaps there is osme latency across your network drives.

This workspace seems to work consistently: Simplefeaturewriterfilecopy.fmwt

Hi Mark! I haven't been able to identify anything in our environment that has changed other than going from 2018 to 2020. We have a workaround with syscaller, but I'm not super comfortable with scripting forced recursive deletes. Will your team be treating this as a regression, analyzing further?


Hi Mark! I haven't been able to identify anything in our environment that has changed other than going from 2018 to 2020. We have a workaround with syscaller, but I'm not super comfortable with scripting forced recursive deletes. Will your team be treating this as a regression, analyzing further?

yes we will try and fix this. It looks like if the output zip file already exists then the entire file is overwritten. If it's a new file then it seems to work OK.


Hi our organization too is experiencing issues with FME Server 2020 and zipped files. We recently upgraded to 2020 from 2018 and have several workspaces that write to zipfiles. The workspace runs as intended in desktop, but when scheduled on server the zipfile is deleted out of the destination folder rather than overwritten and the log returns the warning:

MULTI_WRITER: Skipped adding of some files to zip file "..." because they already exist

 


Hi our organization too is experiencing issues with FME Server 2020 and zipped files. We recently upgraded to 2020 from 2018 and have several workspaces that write to zipfiles. The workspace runs as intended in desktop, but when scheduled on server the zipfile is deleted out of the destination folder rather than overwritten and the log returns the warning:

MULTI_WRITER: Skipped adding of some files to zip file "..." because they already exist

 

@shaundadev, it should be fixed in the latest build (20218), I have yet to download and test it out.

The current workaround uses the command line in the SystemCaller transformer to delete the data before writing, its not ideal, but it works.

rmdir "XXFileLocationXX" /S /Q


@shaundadev, it should be fixed in the latest build (20218), I have yet to download and test it out.

The current workaround uses the command line in the SystemCaller transformer to delete the data before writing, its not ideal, but it works.

rmdir "XXFileLocationXX" /S /Q

Thanks! Deleting the file before writing to it makes it work.


I experience the exact same problem in FME 2019.2. The zip file gets owerwritten and a new file is added to a new zip file resulting in that all existing files in the zip file are lost.


@jan​ , I would submit a ticket to SAFE regarding the issue. My workaround above does solve the issue, but the update version corrected the issue for us (FME(R) 2020.0.3.0 (20200605 - Build 20252 - WIN64))


Reply