Skip to main content
  • EN

I want to simply add a folder with 3 subfolders to an S3 bucket. I'm using the S3Connector transformer and it seems you need a file to Upload to an S3 folder to create a folder, whereas, it seems to me that you can't only add a folder w/o having a file to upload it. I tried adding a csv file to each subfolder to setup my folder structure in S3 and that works to get the main folder w/ 3 subfolders each containing an empty csv file. I then added 3 more S3Connector transformers w/ the Delete option to remove the csv files from each subfolder thinking then I would have the desired folder setup without any files in them. This doesn't work because when you delete the only csv file from the subfolder, it also removes subfolder leaving nothing left in S3.

 

Is there a way to create the desired s3 folder structure without adding a file? Or is it possible to delete the file from the subfolder without removing the subfolder?

Sharing the answer publicly here in case it helps other users running into the same issue.

 

The S3Connector does not currently support creating a folder. Note that this is by-design as S3 has a flat structure instead of a hierarchy as normally seen in a file system (mentioned here in the S3 documentation). S3 supports the folder concept as a way of grouping objects but does not create folders in the usual sense.

 

One possible workaround is to use the PythonCaller and the boto3 module that is already included with an FME install. Alternatively, it may also be possible using the AWS CLI and a SystemCaller.

Reply


Community Stats

30,157
Posts
114,473
Replies
37,802
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.