Skip to main content
Question

Having trouble reading OS Open Names. This product is supplied as multiple (headerless) CSVs packed into multiple sets of zipped folders. I want to create a single reader and assign column names.


Forum|alt.badge.img
Having trouble reading OS Open Names. This product is supplied as multiple (headerless) CSVs packed into multiple sets of zipped folders. I want to create a single reader and assign column names.

4 replies

Forum|alt.badge.img
Typical zip file name ... "opname_csv_nz.zip" ... within each zipped folder the data files I'm after live in a sub-folder called DATA all with names of the form AANN.csv

 

 

Confusingly/annoyingly there is also another sub-folder called DOC which contains CSV files I don't want.

 

 

 

So what I want the reader to do is delve into each of the ZIP folders (there are 9), dig down into the DATA folder in each case and suck out all the CSVs (up to 25 in each DATA folder) and spit them out into a single data stream.

david_r
Celebrity
  • April 9, 2015
Hi,

 

 

consider doing all the unpacking in a startup script. Here are some tips to get you started recursively unpacking a zip-file using Python:

 

 

http://stackoverflow.com/questions/10902409/recursively-unzip-archives-store-filename-extracted-contents-in-dictionary

 

 

David

takashi
Evangelist
  • April 9, 2015
Hi,

 

 

I suppose that the zip contains hierachy foloder structure, but is not recursively zipped.

 

If my understanding is right, this Dataset setting might help you.

 

-----

 

C:\\<directory path>\\*.zip\\**\\DATA\\*.csv

 

-----

 

See also here.

 

Reading from a Zip File (http://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Desktop_Help.htm#../Subsystems/FME_Workbench/Content/Workbench/zip_files_reading.htm)

 

 

Takashi

Forum|alt.badge.img
Thanks Takashi, that sorted it.

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings