Question

Fanning Out and Zipping Files Simultaneously


Userlevel 4
Badge +25

Hey FME'ers. I just wanted to draw your attention to this useful piece of information I found.

See my tweet here: https://twitter.com/FMEEvangelist/status/1151516193263640577

Basically, when doing a Dataset Fanout you can set the folder to be a zip file when you write to it, or you can set the Fanout Expression to have a zip file extension, or you can use both!

I'm not sure when this got implemented. I think the first method has been around for a long time, but the other two might only be in 2019 or newer.

Hope you find this useful!


8 replies

Badge +22

That's quite interesting. Now if only the DataDownload service could do the same...

Userlevel 4
Badge +25

That's quite interesting. Now if only the DataDownload service could do the same...

Can you use this technique on a workspace that's used in Data Download? Hmmm. Apparently not. It doesn't seem to return any output. That's strange. I'll investigate.

Badge +22

Can you use this technique on a workspace that's used in Data Download? Hmmm. Apparently not. It doesn't seem to return any output. That's strange. I'll investigate.

Talk to Rahul about my experiments from the last two weeks, I don't think a case was ever created.

 

 

Lowdown is that any attempt to make dynamic internal folder names in the datadownload service zipfile with a regular file-based writer will end badly.

 

 

And yes I ended up with an empty zipfile a couple of times, I believe when the fanout was a straight attribute.
Userlevel 4
Badge +25

Talk to Rahul about my experiments from the last two weeks, I don't think a case was ever created.

 

 

Lowdown is that any attempt to make dynamic internal folder names in the datadownload service zipfile with a regular file-based writer will end badly.

 

 

And yes I ended up with an empty zipfile a couple of times, I believe when the fanout was a straight attribute.

I'll check with Rahul and make sure something is filed with the devs.

Userlevel 2
Badge +17

Creating multiple zip files with Dataset Fanout has been implemented in 2016 ;-)

Idea: Allow Dataset Fanout to Create Multiple Zip Files

Userlevel 4
Badge +25

Creating multiple zip files with Dataset Fanout has been implemented in 2016 ;-)

Idea: Allow Dataset Fanout to Create Multiple Zip Files

Ha! Was it really that long ago? I just found the enhancement request in our systems recently and it was still open. It was tested and found to work so the request was closed and I thought it would be a good idea to let everyone know! Oh well, I think it sometimes helps to mention these pieces of functionality that not everyone will know about.

Thanks for commenting!

Badge +22

Creating multiple zip files with Dataset Fanout has been implemented in 2016 ;-)

Idea: Allow Dataset Fanout to Create Multiple Zip Files

I knew you could create multiple zipfiles, I had never attempted to create multiple zipfiles within another zip.

Userlevel 2
Badge +17

Creating multiple zip files with Dataset Fanout has been implemented in 2016 ;-)

Idea: Allow Dataset Fanout to Create Multiple Zip Files

Hi @jdh, you are right. If you add the suffix ".zip" to both Destination Fanout Directory and Fanout Expression, the multiple zip files will be saved in a folder whose name is just suffixed by ".zip".

fanout-dataset-and-zip-archiving-1.fmw (FME 2019.1)

A possible way to archive multiplt zip files is to use a FeatureWriter and a File Copy writer.

fanout-dataset-and-zip-archiving-2.fmw (FME 2019.1)

Reply