Hi, is there a way to reference an s3 bucket using an fme server parameter so that we can read or write directly to/from the bucket? I know I can use the S3 uploader, but it would be really handy to be able to use the feature writer instead.
Question
Reference an s3 bucket as a FME server parameter?
This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
An S3 bucket isn't a format, it's just a file container, much like a directory on your PC, so you'll have to first write your data locally in the desired format either using a regular writer or a FeatureWriter before using the S3Uploader. The TempPathnameCreator can be helpful for creating such temporary files since the files created using this transformer automatically gets deleted when FME terminates.
You can user a regular TEXT type published parameter to specify the S3 bucket name used by the S3Uploader.
+7- Contributor
An S3 bucket isn't a format, it's just a file container, much like a directory on your PC, so you'll have to first write your data locally in the desired format either using a regular writer or a FeatureWriter before using the S3Uploader. The TempPathnameCreator can be helpful for creating such temporary files since the files created using this transformer automatically gets deleted when FME terminates.
You can user a regular TEXT type published parameter to specify the S3 bucket name used by the S3Uploader.
thanks.
+9- Safer
Hi. Unfortunately, at this point, AWS S3 buckets mounted as a Resource cannot be accessed from workspaces run from FME Server, unless that access is through an S3-related transformer, such as S3Uploader, S3Downloader, S3Deleter, or S3ObjectLister. The S3 resources mount is really just to enable backup workflows. There is an open request to enable access to the S3 resource mounts from within FME Workbench which I will add you to.
+7- Contributor
Hi. Unfortunately, at this point, AWS S3 buckets mounted as a Resource cannot be accessed from workspaces run from FME Server, unless that access is through an S3-related transformer, such as S3Uploader, S3Downloader, S3Deleter, or S3ObjectLister. The S3 resources mount is really just to enable backup workflows. There is an open request to enable access to the S3 resource mounts from within FME Workbench which I will add you to.
Hi. Unfortunately, at this point, AWS S3 buckets mounted as a Resource cannot be accessed from workspaces run from FME Server, unless that access is through an S3-related transformer, such as S3Uploader, S3Downloader, S3Deleter, or S3ObjectLister. The S3 resources mount is really just to enable backup workflows. There is an open request to enable access to the S3 resource mounts from within FME Workbench which I will add you to.
https://knowledge.safe.com/idea/22137/ability-to-mount-s3-bucket-as-a-drive-from-within.html
+7- Contributor
Hi. Unfortunately, at this point, AWS S3 buckets mounted as a Resource cannot be accessed from workspaces run from FME Server, unless that access is through an S3-related transformer, such as S3Uploader, S3Downloader, S3Deleter, or S3ObjectLister. The S3 resources mount is really just to enable backup workflows. There is an open request to enable access to the S3 resource mounts from within FME Workbench which I will add you to.
+7- Contributor
@johnglick - If the file is already in s3 use the s3 downloader, put it in temp resources location - use a feature reader, manipulate and upload using s3 uploader.
you can't manipulate the file while it is in s3