Hi, is there a way to reference an s3 bucket using an fme server parameter so that we can read or write directly to/from the bucket? I know I can use the S3 uploader, but it would be really handy to be able to use the feature writer instead.
An S3 bucket isn't a format, it's just a file container, much like a directory on your PC, so you'll have to first write your data locally in the desired format either using a regular writer or a FeatureWriter before using the S3Uploader. The TempPathnameCreator can be helpful for creating such temporary files since the files created using this transformer automatically gets deleted when FME terminates.
You can user a regular TEXT type published parameter to specify the S3 bucket name used by the S3Uploader.
+1- Contributor
david_r wrote:
An S3 bucket isn't a format, it's just a file container, much like a directory on your PC, so you'll have to first write your data locally in the desired format either using a regular writer or a FeatureWriter before using the S3Uploader. The TempPathnameCreator can be helpful for creating such temporary files since the files created using this transformer automatically gets deleted when FME terminates.
You can user a regular TEXT type published parameter to specify the S3 bucket name used by the S3Uploader.
thanks.
+5- Safer
Hi. Unfortunately, at this point, AWS S3 buckets mounted as a Resource cannot be accessed from workspaces run from FME Server, unless that access is through an S3-related transformer, such as S3Uploader, S3Downloader, S3Deleter, or S3ObjectLister. The S3 resources mount is really just to enable backup workflows. There is an open request to enable access to the S3 resource mounts from within FME Workbench which I will add you to.
+1- Contributor
stewartatsafe wrote:
Hi. Unfortunately, at this point, AWS S3 buckets mounted as a Resource cannot be accessed from workspaces run from FME Server, unless that access is through an S3-related transformer, such as S3Uploader, S3Downloader, S3Deleter, or S3ObjectLister. The S3 resources mount is really just to enable backup workflows. There is an open request to enable access to the S3 resource mounts from within FME Workbench which I will add you to.
stewartatsafe wrote:
Hi. Unfortunately, at this point, AWS S3 buckets mounted as a Resource cannot be accessed from workspaces run from FME Server, unless that access is through an S3-related transformer, such as S3Uploader, S3Downloader, S3Deleter, or S3ObjectLister. The S3 resources mount is really just to enable backup workflows. There is an open request to enable access to the S3 resource mounts from within FME Workbench which I will add you to.
https://knowledge.safe.com/idea/22137/ability-to-mount-s3-bucket-as-a-drive-from-within.html
+6stewartatsafe wrote:
Hi. Unfortunately, at this point, AWS S3 buckets mounted as a Resource cannot be accessed from workspaces run from FME Server, unless that access is through an S3-related transformer, such as S3Uploader, S3Downloader, S3Deleter, or S3ObjectLister. The S3 resources mount is really just to enable backup workflows. There is an open request to enable access to the S3 resource mounts from within FME Workbench which I will add you to.
+6@johnglick - If the file is already in s3 use the s3 downloader, put it in temp resources location - use a feature reader, manipulate and upload using s3 uploader.
you can't manipulate the file while it is in s3
Reply
Rich Text Editor, editor1
Editor toolbars
Press ALT 0 for help
Helpful Members This Week
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.