Skip to main content
Question

Fanning Out and Zipping Files Simultaneously

  • July 17, 2019
  • 8 replies
  • 131 views
mark2atsafe
Safer
Forum|alt.badge.img+56

Hey FME'ers. I just wanted to draw your attention to this useful piece of information I found.

See my tweet here: https://twitter.com/FMEEvangelist/status/1151516193263640577

Basically, when doing a Dataset Fanout you can set the folder to be a zip file when you write to it, or you can set the Fanout Expression to have a zip file extension, or you can use both!

I'm not sure when this got implemented. I think the first method has been around for a long time, but the other two might only be in 2019 or newer.

Hope you find this useful!

FME Evangelist to the Rich and Famous!!
This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

8 replies

jdh
Contributor
Forum|alt.badge.img+37
  • jdhContributor
  • Contributor
  • 2002 replies
  • July 17, 2019

That's quite interesting. Now if only the DataDownload service could do the same...

mark2atsafe
Safer
Forum|alt.badge.img+56
  • mark2atsafeSafer
  • Author
  • Safer
  • 2554 replies
  • July 17, 2019

That's quite interesting. Now if only the DataDownload service could do the same...

Can you use this technique on a workspace that's used in Data Download? Hmmm. Apparently not. It doesn't seem to return any output. That's strange. I'll investigate.

FME Evangelist to the Rich and Famous!!
jdh
Contributor
Forum|alt.badge.img+37
  • jdhContributor
  • Contributor
  • 2002 replies
  • July 17, 2019

Can you use this technique on a workspace that's used in Data Download? Hmmm. Apparently not. It doesn't seem to return any output. That's strange. I'll investigate.

Talk to Rahul about my experiments from the last two weeks, I don't think a case was ever created.

 

 

Lowdown is that any attempt to make dynamic internal folder names in the datadownload service zipfile with a regular file-based writer will end badly.

 

 

And yes I ended up with an empty zipfile a couple of times, I believe when the fanout was a straight attribute.
mark2atsafe
Safer
Forum|alt.badge.img+56
  • mark2atsafeSafer
  • Author
  • Safer
  • 2554 replies
  • July 17, 2019

Talk to Rahul about my experiments from the last two weeks, I don't think a case was ever created.

 

 

Lowdown is that any attempt to make dynamic internal folder names in the datadownload service zipfile with a regular file-based writer will end badly.

 

 

And yes I ended up with an empty zipfile a couple of times, I believe when the fanout was a straight attribute.

I'll check with Rahul and make sure something is filed with the devs.

FME Evangelist to the Rich and Famous!!
takashi
Celebrity
  • takashiCelebrity
  • 7843 replies
  • July 17, 2019

Creating multiple zip files with Dataset Fanout has been implemented in 2016 ;-)

Idea: Allow Dataset Fanout to Create Multiple Zip Files

Why not inspect features with Visual/Data Preview and Feature/Record Information before writing them into a destination dataset?
mark2atsafe
Safer
Forum|alt.badge.img+56
  • mark2atsafeSafer
  • Author
  • Safer
  • 2554 replies
  • July 17, 2019

Creating multiple zip files with Dataset Fanout has been implemented in 2016 ;-)

Idea: Allow Dataset Fanout to Create Multiple Zip Files

Ha! Was it really that long ago? I just found the enhancement request in our systems recently and it was still open. It was tested and found to work so the request was closed and I thought it would be a good idea to let everyone know! Oh well, I think it sometimes helps to mention these pieces of functionality that not everyone will know about.

Thanks for commenting!

FME Evangelist to the Rich and Famous!!
jdh
Contributor
Forum|alt.badge.img+37
  • jdhContributor
  • Contributor
  • 2002 replies
  • July 17, 2019

Creating multiple zip files with Dataset Fanout has been implemented in 2016 ;-)

Idea: Allow Dataset Fanout to Create Multiple Zip Files

I knew you could create multiple zipfiles, I had never attempted to create multiple zipfiles within another zip.

takashi
Celebrity
  • takashiCelebrity
  • 7843 replies
  • July 17, 2019

Creating multiple zip files with Dataset Fanout has been implemented in 2016 ;-)

Idea: Allow Dataset Fanout to Create Multiple Zip Files

Hi @jdh, you are right. If you add the suffix ".zip" to both Destination Fanout Directory and Fanout Expression, the multiple zip files will be saved in a folder whose name is just suffixed by ".zip".

fanout-dataset-and-zip-archiving-1.fmw (FME 2019.1)

A possible way to archive multiplt zip files is to use a FeatureWriter and a File Copy writer.

fanout-dataset-and-zip-archiving-2.fmw (FME 2019.1)

Why not inspect features with Visual/Data Preview and Feature/Record Information before writing them into a destination dataset?

Community Stats

32,080
Posts
121,818
Replies
39,714
Members

Latest FME

Terms & ConditionsCookie settingsAccessibility statement
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.