Thank you!
Courtney
Solved
Uploading Transformed Data to AmazonS3 Without Writing Locally
+21- Safer
Is it possible to upload data, that has been modified, to Amazon S3 from within the workspace that is modifying the data? For example, I have a workspace that reads in single-band landsat imagery (as GeoTIFFs), processes the images, and combines them into composite images. Instead of writing the combined-band images locally, I want to upload them to Amazon S3. I tried using the S3Uploader in the same workspace, after the image processing transformers, but the data source options for the S3Uploader seem to be limited to existing files or expressions - I ended up uploading the original single-band images, rather than the processed images... Is it possible to upload the processed images directly to AmazonS3 without having to write them locally first (using FME Desktop)?
Best answer by takashi
Hi Courtney,
I don't think it's possible to upload a file to S3 without saving it once into the local disk. If your goal is to automate the conjunction of translation and uploading, another workspace containing a WorkspaceRunner could be a workaround.
The WorkspaceRunner runs the existing workspace, which reads the source rasters, transforms them, and writes the resulting image into a local folder. Set "Yes" to the "Wait for Job to Complete" parameter of the WorkspaceRunner.
You can then use an S3Uploader connected to the Succeeded port of the WorkspaceRunner, to upload the resulting image after completing the translation.
Takashi
This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
- Best Answer
Hi Courtney,
I don't think it's possible to upload a file to S3 without saving it once into the local disk. If your goal is to automate the conjunction of translation and uploading, another workspace containing a WorkspaceRunner could be a workaround.
The WorkspaceRunner runs the existing workspace, which reads the source rasters, transforms them, and writes the resulting image into a local folder. Set "Yes" to the "Wait for Job to Complete" parameter of the WorkspaceRunner.
You can then use an S3Uploader connected to the Succeeded port of the WorkspaceRunner, to upload the resulting image after completing the translation.
Takashi
This is informative.
Just wanted to check if it is still the case of writing the output to the local disk and upload to s3 or there's been an upgrade ?
Thanks.
+62- Celebrity
This is informative.
Just wanted to check if it is still the case of writing the output to the local disk and upload to s3 or there's been an upgrade ?
Thanks.