We have a need to read some spatial data that is zipped up, but several of the zips are then nested inside another zip, just to make things fun. Like this:
c:\\temp\\top.zip\\1.zip\\1.gml
c:\\temp\\top.zip\\2.zip\\2.gml
FME can read say GML from inside a Zip, but not GML from a Zip inside a Zip. Anyone got any magic to unpack the top level Zip? I suspect the answer is Python, perhaps as either a PythonCaller or Startup script and then I could use a FeatureReader to get at the data at the next level.
Seems a bit contrived but its a genuine requirement as its the way a product is shipped. 1 zip file was clearly not enough!
# Script Example for PythonCreator # Extract zip files nested in a zip file, # create features containing an attribute that stores extracted zip file path. # Not recursive. Applicable to just one level nesting. import fmeobjects, zipfile, os class NestedZipUnpacker(object): def close(self): folder = FME_MacroValues['OUTPUT_FOLDER_PATH'] try: # Extract all files archived in the spacified zip file. with zipfile.ZipFile(FME_MacroValues['ZIPFILE_PATH']) as z: z.extractall(folder) # Create features for each extracted zip file path. for path in [os.path.join(folder, fname) for fname in os.listdir(folder)]: if zipfile.is_zipfile(path): feature = fmeobjects.FMEFeature() feature.setAttribute('_zip_path', path) self.pyoutput(feature) except Exception as ex: logger = fmeobjects.FMELogFile() logger.logMessageString('%s' % ex, fmeobjects.FME_ERROR)
Assume that these two user parameters are defined in the workspace.
OUTPUT_FOLDER_PATH: existing folder path into which the extracted zip files will be saved
ZIPFILE_PATH: the top level zip file path
FYI.
This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
# Script Example for PythonCreator # Extract zip files nested in a zip file, # create features containing an attribute that stores extracted zip file path. # Not recursive. Applicable to just one level nesting. import fmeobjects, zipfile, os class NestedZipUnpacker(object): def close(self): folder = FME_MacroValues['OUTPUT_FOLDER_PATH'] try: # Extract all files archived in the spacified zip file. with zipfile.ZipFile(FME_MacroValues['ZIPFILE_PATH']) as z: z.extractall(folder) # Create features for each extracted zip file path. for path in [os.path.join(folder, fname) for fname in os.listdir(folder)]: if zipfile.is_zipfile(path): feature = fmeobjects.FMEFeature() feature.setAttribute('_zip_path', path) self.pyoutput(feature) except Exception as ex: logger = fmeobjects.FMELogFile() logger.logMessageString('%s' % ex, fmeobjects.FME_ERROR)
Assume that these two user parameters are defined in the workspace.
OUTPUT_FOLDER_PATH: existing folder path into which the extracted zip files will be saved
# Script Example for PythonCreator # Extract zip files nested in a zip file, # create features containing an attribute that stores extracted zip file path. # Not recursive. Applicable to just one level nesting. import fmeobjects, zipfile, os class NestedZipUnpacker(object): def close(self): folder = FME_MacroValues['OUTPUT_FOLDER_PATH'] try: # Extract all files archived in the spacified zip file. with zipfile.ZipFile(FME_MacroValues['ZIPFILE_PATH']) as z: z.extractall(folder) # Create features for each extracted zip file path. for path in [os.path.join(folder, fname) for fname in os.listdir(folder)]: if zipfile.is_zipfile(path): feature = fmeobjects.FMEFeature() feature.setAttribute('_zip_path', path) self.pyoutput(feature) except Exception as ex: logger = fmeobjects.FMELogFile() logger.logMessageString('%s' % ex, fmeobjects.FME_ERROR)
Assume that these two user parameters are defined in the workspace.
OUTPUT_FOLDER_PATH: existing folder path into which the extracted zip files will be saved
ZIPFILE_PATH: the top level zip file path
FYI.
Inspired from this discussion, published a custom transformer named ZipExtractor in the FME Store.
