Question

Write GML to multiple ZIP files

  • 10 September 2015
  • 4 replies
  • 4 views

Badge +14
Hi there Community,

 

 

I'm trying to fanout GML files into multiple ZIP files, where the GML file always has the same name but the ZIP file name is controlled by an attribute. So I want:

 

 

test.gml\\North.zip

 

test.gml\\South.zip

 

test.gml\\East.zip

 

test.gml\\West.zip

 

 

It appears that the fanout dataset capability doesn't allow me to do this and I can only create many differently named GML files in the same ZIP.

 

 

Does anyone know a way around this?

 

 

Thanks in advance, Dave

4 replies

Badge +14
...and perhaps I should say there is also a little extra complexity in that I am currently using the file copy writer to also copy a couple of standard supporting files across into each zip file. So if this ends up as a Shutdown Script solution I'll also need to have scope to add some common files into the zip too!
Userlevel 4
Hi Dave

 

 

I tried the following, which in theory, ought to work (according to me ;-)

 

 

 

 

But it didn't, I was only able to write one single zip file, e.g. "C:\\Temp\\West.zip". If you look at the log, it seems FME re-uses the same temp zip file without considering the fanout, so it all ends ut in the same output zip. Sounds like a bug / unwanted feature to me.

 

 

Looks like you'll have to break out a Python script or possibly do something with a WorkspaceRunner.

 

 

David
Userlevel 2
Badge +17
Hi Dave and David,

 

 

Last year, I also found the Dataset Fanout does not support zip compressing, and sent a question to Safe support. Unfortunately the current FME doesn't support fanout with zip.

 

 

The Workbench help says: "Note: You cannot create multiple zip files using Dataset Fanout (therefore, you cannot set the Fanout Suffix parameter to .zip)."

 

-- FME Workbench > Readers and Writers > Readers and Writers: Zip Files and URLs > Writing to a Zip File

 

 

This is an example for writing zip files in Shutdown Python Script.

 

-----

 

#Shutdown Python Script Example

 

import fme, os, zipfile

 

 

# You will have to define these two lists appropriately.

 

baseNames = ['North', 'South', 'East', 'West']

 

commonFiles = ['C:\\\\Data\\\\common1.txt', 'C:\\\\Data\\\\common2.txt']

 

 

# Get GML and ZIP folder paths from published parameters.

 

gmlDir = fme.macroValues['DestDataset_GML']

 

zipDir = fme.macroValues['ZIP_ROOT_FOLDER']

 

 

# If the ZIP folder doesn't exist, create it.

 

if not os.path.exists(zipDir):

 

    os.makedirs(zipDir)

 

 

try:

 

    for name in baseNames:

 

        # GML file path

 

        gmlPath = '%s\\\\%s.gml' % (gmlDir, name)

 

        if not os.path.exists(gmlPath):

 

            # If the GML file doesn't exist, what should I do?

 

            continue

 

        

 

        # ZIP file path

 

        zipPath = '%s\\\\%s.zip' % (zipDir, name)

 

        

 

        # Write the GML file into a new ZIP file.

 

        z = zipfile.ZipFile(zipPath, 'w', zipfile.ZIP_DEFLATED)

 

        z.write(gmlPath, '%s.gml' % name)

 

        z.close()

 

        

 

        # Append the common files into the ZIP file.

 

        z = zipfile.ZipFile(zipPath, 'a', zipfile.ZIP_DEFLATED)

 

        for common in commonFiles:

 

            z.write(common, os.path.basename(common))

 

        z.close()

 

except Exception as ex:

 

    # Do something if necessary.

 

    pass

 

finally:

 

    # Do something if necessary.

 

    pass

 

-----

 

 

Hope this helps.

 

 

Takashi
Badge +14
Hi Takashi/David - Thanks guys. For the time being I've resolved this with a WorkspaceRunner but hit this problem with that solution https://knowledge.safe.com/communityanswers?feedtype=RECENT&dc=All&criteria=ALLQUESTIONS#!/feedtype=SINGLE_QUESTION_SEARCH_RESULT&id=906a0000000dAxiAAE (https://knowledge.safe.com/communityanswers?feedtype=RECENT&dc=All&criteria=ALLQUESTIONS#!/feedtype=SINGLE_QUESTION_SEARCH_RESULT&id=906a0000000dAxiAAE) oh the joys of working in IT. I like the Python approach though so am going to pursue that further to see if I can get it to work. Thanks for sending that template Takashi, much appreciated. A feel a feature request coming on!

 

 

Cheers, Dave

Reply