Skip to main content

I have multiple shapefiles (total size is 8.4 GB). Those need to be zipped. When using FME's ZipArchiver, compressed file size is 4.43 GB. But when using third party app – 7zip, the result file is ~11% smaller (3.94 GB). My guess woul be that ZipArchiver uses different libararies for compression. Maybe it is possible to use 7zip libraries in ZipArchiver, when compressing files?

LZMA compression won't work in this case, because I need to be able to extract archives with built-in Windows extractor.

Hi @fikusas, the ZipArchiver uses the native Python zipfile module to perform archiving, and I don't have any plan to change it to use 7-Zip.

In Python 3.6 and earlier, the zipfile module doesn't have any options to control compression level.

In Python 3.7 or later, the zipfile.ZipFile class has complesslevel parameter. If the workspace is configured to run with Python 3.7+, I think you can modify the script of the PythonCaller within the transformer to control the compression level. See here to learn more:

ZipFile Objects - Python Documentation

Another option, if you prefer to use 7-Zip to compress files, you can use the SystemCaller to execute 7z.exe which is the command line version of 7-Zip. See here to learn more:

Command Line Version User's Guide - 7z.exe


Hi @fikusas, the ZipArchiver uses the native Python zipfile module to perform archiving, and I don't have any plan to change it to use 7-Zip.

In Python 3.6 and earlier, the zipfile module doesn't have any options to control compression level.

In Python 3.7 or later, the zipfile.ZipFile class has complesslevel parameter. If the workspace is configured to run with Python 3.7+, I think you can modify the script of the PythonCaller within the transformer to control the compression level. See here to learn more:

ZipFile Objects - Python Documentation

Another option, if you prefer to use 7-Zip to compress files, you can use the SystemCaller to execute 7z.exe which is the command line version of 7-Zip. See here to learn more:

Command Line Version User's Guide - 7z.exe

I added "Compression Level" parameter to the ZipArchiver. You can download the new one from FME Hub site.

https://hub.safe.com/publishers/pacific-spatial-solutions/transformers/ziparchiver

Just be aware that the parameter works only if the version of Python interpreter is 3.7 or later. If the Python version is 3.6 or earlier, the parameters setting will just be ignored.

----------

Compression Level (Python 3.7+) :

If FME uses Python 3.7+ when running the workspace, you can control the compression levels with this parameter.

  • Level 0 indicates just archiving with no compression.
  • In a level between 1-9, less number indicates faster and less compression, greater number indicates slower and more compression. However, the order of the numbers does not always exactly match the order of actual compression levels. Depending on the conditions, 6 could be more compression than 9 for example.
  • Currently (in Python 3.7), "Default" is equivalent to level 6, but it may be changed in the future.

This parameter works only with Python 3.7 or later. If the version of Python interpreter is 3.6 or earlier, this parameter setting will just be ignored.


Thanks for your effort, @takashi! compresslevel of 9 doesn't gives as good compression as using 7zip. SystemCaller worked just fine zipping via 7z.exe.


Reply