Skip to main content
Solved

S3Connector and download only folders that contain 2025


peterhearns
Contributor
Forum|alt.badge.img+5

Good morning,

I have a folder in an S3 bucket that contains a folder for each day of the year and each of those folders contain about 24 files. Since I need to download the data in order to process it, is there a way to only download the folders that contain the word “2025”?  I have not been able to format a wildcard that works for a folder. I could also potentially use a wildcard to select the files, if there is a way to do that.

Thank you,

Peter

Best answer by david_r

I share the opinion of ​@carlm, you should use the option Include subfolders = No to only read the first level, i.e. the folder names, then filter (e.g Tester) before reading the individual folder contents.

View original
Did this help you find an answer to your question?

3 replies

carlm
Contributor
Forum|alt.badge.img+13
  • Contributor
  • June 5, 2025

You will need to use the List function of the S3Connector first, use a tester to filter the file containing the 2025 file and then Download the file that you need.


david_r
Celebrity
  • Best Answer
  • June 5, 2025

I share the opinion of ​@carlm, you should use the option Include subfolders = No to only read the first level, i.e. the folder names, then filter (e.g Tester) before reading the individual folder contents.


peterhearns
Contributor
Forum|alt.badge.img+5
  • Author
  • Contributor
  • June 5, 2025

Excellent, thank you @david_r and @carlm for the quick responses.  This worked perfectly and now I don’t fill up my local drive!  I wish I could select both as the Best Answer.  


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings