Skip to main content
Solved

Download zip files from Google Cloud bucket

  • June 24, 2025
  • 4 replies
  • 93 views
kennethrichards
Contributor
Forum|alt.badge.img+8

I am trying to set up a process that downloads new zip files containing shapefiles from a Google Bucket. The bucket contains four folders at the top level and in each of those folders are folders for individual project areas. In those project folders there are mp4s, pdfs and zip files with the shape files. I would like to just download those zip files, and keep them grouped by project, and none of the others every time the flow is triggered. So far I have figured out out how to list the files and filter out everything else not a zip file. The list is great but I want to download them and not just list them. This is my first time working with Google Cloud Storage buckets.

 

Best answer by kennethrichards

Have a good read if the Doc. 

https://docs.safe.com/fme/html/FME-Form-Documentation/FME-Transformers/Transformers/googlecloudstorageconnector-pkg.htm

 

Now that you've filtered for zips you should be able to feed the path of the zip file in as the path for the Path to Download in a new Connector. 

You might want to define an Output folder where all zips will get downloaded too. If you need to keep the folder structure of the online storage then you should be able to combine the path of your output folder with the path on the cloud. 

 

You want to use the download as file option rather than folder in your case. 

 

Use a sampler to test with only a handful of zips before you run the whole thing to make sure it's working as expected

How do I feed that path into the second GoogleCloudStorageConnector? This connector has got me stumped. I have fed the sampled list into the connector and then set it to download as a file to a local test folder and I am not getting anything to download in the destination folder. Any suggestions?

 

danilo_fme
This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

4 replies

j.botterill
Influencer
Forum|alt.badge.img+53
  • j.botterillInfluencer
  • Influencer
  • June 25, 2025

There are some articles out there that might help https://fme.safe.com/blog/2020/04/connect-google-cloud-fme/ and https://hub.safe.com/publishers/safe/packages/google-cloud-storage

Typical uses for the GoogleCloudStorageConnector:

  • manage datasets on Google Cloud Storage by uploading, downloading, and deleting files and folders
  • transfer a file's contents (such as XML or raster) into or out of an attribute in FME
  • read downloaded Google Cloud Storage data using the FeatureReader, or upload data written by the FeatureWriter to Google Cloud Storage
  • retrieve file and folder names, paths, links and other information from Google Cloud Storage to use elsewhere in a workspace

Supported Python Versions: 3.8+

virtualcitymatt
Celebrity
Forum|alt.badge.img+47

Have a good read if the Doc. 

https://docs.safe.com/fme/html/FME-Form-Documentation/FME-Transformers/Transformers/googlecloudstorageconnector-pkg.htm

 

Now that you've filtered for zips you should be able to feed the path of the zip file in as the path for the Path to Download in a new Connector. 

You might want to define an Output folder where all zips will get downloaded too. If you need to keep the folder structure of the online storage then you should be able to combine the path of your output folder with the path on the cloud. 

 

You want to use the download as file option rather than folder in your case. 

 

Use a sampler to test with only a handful of zips before you run the whole thing to make sure it's working as expected

kennethrichards
Contributor
Forum|alt.badge.img+8
  • kennethrichardsContributor
  • Author
  • Contributor
  • Best Answer
  • June 25, 2025

Have a good read if the Doc. 

https://docs.safe.com/fme/html/FME-Form-Documentation/FME-Transformers/Transformers/googlecloudstorageconnector-pkg.htm

 

Now that you've filtered for zips you should be able to feed the path of the zip file in as the path for the Path to Download in a new Connector. 

You might want to define an Output folder where all zips will get downloaded too. If you need to keep the folder structure of the online storage then you should be able to combine the path of your output folder with the path on the cloud. 

 

You want to use the download as file option rather than folder in your case. 

 

Use a sampler to test with only a handful of zips before you run the whole thing to make sure it's working as expected

How do I feed that path into the second GoogleCloudStorageConnector? This connector has got me stumped. I have fed the sampled list into the connector and then set it to download as a file to a local test folder and I am not getting anything to download in the destination folder. Any suggestions?

 

j.botterill
Influencer
Forum|alt.badge.img+53
  • j.botterillInfluencer
  • Influencer
  • June 25, 2025

Following the connector transformer,  which currently has an output called _download_path, place a featureReader

read downloaded Google Cloud Storage data using the FeatureReader, or upload data written by the FeatureWriter to Google Cloud Storage

With the FeatureReader, specify the main format you are downloading from GoogleCloudStorage and the dataset will be _download_path attribute, selecting the down arrow to find the initator attributes

 

Community Stats

32,129
Posts
121,967
Replies
39,779
Members

Latest FME

Terms & ConditionsCookie settingsAccessibility statement
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.