Skip to main content
Solved

S3Connector and download only folders that contain 2025

  • June 5, 2025
  • 3 replies
  • 33 views
peterhearns
Contributor
Forum|alt.badge.img+7

Good morning,

I have a folder in an S3 bucket that contains a folder for each day of the year and each of those folders contain about 24 files. Since I need to download the data in order to process it, is there a way to only download the folders that contain the word “2025”?  I have not been able to format a wildcard that works for a folder. I could also potentially use a wildcard to select the files, if there is a way to do that.

Thank you,

Peter

Best answer by david_r

I share the opinion of ​@carlm, you should use the option Include subfolders = No to only read the first level, i.e. the folder names, then filter (e.g Tester) before reading the individual folder contents.

This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

3 replies

carlm
Contributor
Forum|alt.badge.img+16
  • carlmContributor
  • Contributor
  • June 5, 2025

You will need to use the List function of the S3Connector first, use a tester to filter the file containing the 2025 file and then Download the file that you need.

FME can do it, I'm having a beer! Cheers
david_r
Celebrity
  • david_rCelebrity
  • Best Answer
  • June 5, 2025

I share the opinion of ​@carlm, you should use the option Include subfolders = No to only read the first level, i.e. the folder names, then filter (e.g Tester) before reading the individual folder contents.

peterhearns
Contributor
Forum|alt.badge.img+7
  • peterhearnsContributor
  • Author
  • Contributor
  • June 5, 2025

Excellent, thank you @david_r and @carlm for the quick responses.  This worked perfectly and now I don’t fill up my local drive!  I wish I could select both as the Best Answer.  

Community Stats

32,129
Posts
121,967
Replies
39,779
Members

Latest FME

Terms & ConditionsCookie settingsAccessibility statement
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.