We have a need to read some spatial data that is zipped up, but several of the zips are then nested inside another zip, just to make things fun. Like this:
c:\\temp\\top.zip\\1.zip\\1.gml
c:\\temp\\top.zip\\2.zip\\2.gml
FME can read say GML from inside a Zip, but not GML from a Zip inside a Zip. Anyone got any magic to unpack the top level Zip? I suspect the answer is Python, perhaps as either a PythonCaller or Startup script and then I could use a FeatureReader to get at the data at the next level.
Seems a bit contrived but its a genuine requirement as its the way a product is shipped. 1 zip file was clearly not enough!
# Script Example for PythonCreator # Extract zip files nested in a zip file, # create features containing an attribute that stores extracted zip file path. # Not recursive. Applicable to just one level nesting. import fmeobjects, zipfile, os class NestedZipUnpacker(object): def close(self): folder = FME_MacroValues['OUTPUT_FOLDER_PATH'] try: # Extract all files archived in the spacified zip file. with zipfile.ZipFile(FME_MacroValues['ZIPFILE_PATH']) as z: z.extractall(folder) # Create features for each extracted zip file path. for path in [os.path.join(folder, fname) for fname in os.listdir(folder)]: if zipfile.is_zipfile(path): feature = fmeobjects.FMEFeature() feature.setAttribute('_zip_path', path) self.pyoutput(feature) except Exception as ex: logger = fmeobjects.FMELogFile() logger.logMessageString('%s' % ex, fmeobjects.FME_ERROR)
Assume that these two user parameters are defined in the workspace.
OUTPUT_FOLDER_PATH: existing folder path into which the extracted zip files will be saved
This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
# Script Example for PythonCreator # Extract zip files nested in a zip file, # create features containing an attribute that stores extracted zip file path. # Not recursive. Applicable to just one level nesting. import fmeobjects, zipfile, os class NestedZipUnpacker(object): def close(self): folder = FME_MacroValues['OUTPUT_FOLDER_PATH'] try: # Extract all files archived in the spacified zip file. with zipfile.ZipFile(FME_MacroValues['ZIPFILE_PATH']) as z: z.extractall(folder) # Create features for each extracted zip file path. for path in [os.path.join(folder, fname) for fname in os.listdir(folder)]: if zipfile.is_zipfile(path): feature = fmeobjects.FMEFeature() feature.setAttribute('_zip_path', path) self.pyoutput(feature) except Exception as ex: logger = fmeobjects.FMELogFile() logger.logMessageString('%s' % ex, fmeobjects.FME_ERROR)
Assume that these two user parameters are defined in the workspace.
OUTPUT_FOLDER_PATH: existing folder path into which the extracted zip files will be saved
# Script Example for PythonCreator # Extract zip files nested in a zip file, # create features containing an attribute that stores extracted zip file path. # Not recursive. Applicable to just one level nesting. import fmeobjects, zipfile, os class NestedZipUnpacker(object): def close(self): folder = FME_MacroValues['OUTPUT_FOLDER_PATH'] try: # Extract all files archived in the spacified zip file. with zipfile.ZipFile(FME_MacroValues['ZIPFILE_PATH']) as z: z.extractall(folder) # Create features for each extracted zip file path. for path in [os.path.join(folder, fname) for fname in os.listdir(folder)]: if zipfile.is_zipfile(path): feature = fmeobjects.FMEFeature() feature.setAttribute('_zip_path', path) self.pyoutput(feature) except Exception as ex: logger = fmeobjects.FMELogFile() logger.logMessageString('%s' % ex, fmeobjects.FME_ERROR)
Assume that these two user parameters are defined in the workspace.
OUTPUT_FOLDER_PATH: existing folder path into which the extracted zip files will be saved
ZIPFILE_PATH: the top level zip file path
FYI.
Inspired from this discussion, published a custom transformer named ZipExtractor in the FME Store.
