Can I read shape files dat are gzipped (.gz)?

Hi @robbie_botha

I was wondering if you had found a solution to this issue. I need to read hundreds of thousands of shapefiles which are gzipped on a server and load them into PostGIS.

Thanks!

Best regards,

Olivier

Yes, the GzipDecompressor can be used to decompress a gzip file, but in FME 2016.1+. I'm afraid it won't work in your FME version 2015. Also, the transformer doesn't support extracting files from a tar archive. A Shapefile dataset consists of 3 or more files, so possibly your data could be a *.tar.gz file.

As a workaround, if your platform is Windows, you can use a free software called 7-Zip (7z.exe) through the SystemCaller transformer.

Firstly donwnload 7-Zip from here and install it into your machine.

Example 1 (Command Line in SystemCaller): Decompress a gz file

""C:\Program Files\7-Zip\7z.exe" e -tgzip "C:\FME\tmp\data\file.txt.gz" -o"C:\FME\tmp\data\a" -aoa"

Example 2 (Use two SystemCallers): Decompress a tar.gz file and extract files from the tar.
1st SystemCaller (decompress tar.gz)

""C:\Program Files\7-Zip\7z.exe" e -tgzip "C:\FME\tmp\data\file.tar.gz" -o"C:\FME\tmp\data\b" -aoa"

2nd SystemCaller (extract files from tar)

""C:\Program Files\7-Zip\7z.exe" e -ttar "C:\FME\tmp\data\b\file.tar" -o"C:\FME\tmp\data\b" -aoa"

 See the help of 7-Zip to learn more.

Thank you Takashi and Danilo! However I was looking for a way to read gzip (.gz) files without decompressing them, in a way similar to the zip option of the Shape reader. In my case each individual shape subfile is gzipped. Decompressing them would take a lot of disk space.

There is no way to read file without decompressing them. Even if the compression format was 'zip', FME extracts the zip file and save the extracted files into the temporary folder, and then reads them. Since the unzipped files will be removed automatically, looks like FME reads the zip file directly.

 

If you want to save disk space, consider using the TempPathnameCreator to generate folder or file paths to which decompressed files are saved. FME will removes automatically the folders/files with the paths generated by the transformer after the translation has completed.

 

 

Thank you Takashi and Danilo! However I was looking for a way to read gzip (.gz) files without decompressing them, in a way similar to the zip option of the Shape reader. In my case each individual shape subfile is gzipped. Decompressing them would take a lot of disk space.

This is also possible.

 

 

 

This is also possible.

 

 

 

Hi Takashi,

 

 

I have tried the following in an attempt to make a gzip reader for shape files (without the copying and manual unzipping). However the FeatureReader rejects the path because it probably no longer exists at that process step:

 

So you see what I should do in order to make this work?

 

Thanks!

 

Best regards,

 

Olivier