Thank you!
Courtney
Solved
Uploading Transformed Data to AmazonS3 Without Writing Locally
+18- Safer
Is it possible to upload data, that has been modified, to Amazon S3 from within the workspace that is modifying the data? For example, I have a workspace that reads in single-band landsat imagery (as GeoTIFFs), processes the images, and combines them into composite images. Instead of writing the combined-band images locally, I want to upload them to Amazon S3. I tried using the S3Uploader in the same workspace, after the image processing transformers, but the data source options for the S3Uploader seem to be limited to existing files or expressions - I ended up uploading the original single-band images, rather than the processed images... Is it possible to upload the processed images directly to AmazonS3 without having to write them locally first (using FME Desktop)?
Best answer by takashi
Hi Courtney,
View original
I don't think it's possible to upload a file to S3 without saving it once into the local disk. If your goal is to automate the conjunction of translation and uploading, another workspace containing a WorkspaceRunner could be a workaround.
The WorkspaceRunner runs the existing workspace, which reads the source rasters, transforms them, and writes the resulting image into a local folder. Set "Yes" to the "Wait for Job to Complete" parameter of the WorkspaceRunner.
You can then use an S3Uploader connected to the Succeeded port of the WorkspaceRunner, to upload the resulting image after completing the translation.
Takashi
- Best Answer
Hi Courtney,
I don't think it's possible to upload a file to S3 without saving it once into the local disk. If your goal is to automate the conjunction of translation and uploading, another workspace containing a WorkspaceRunner could be a workaround.
The WorkspaceRunner runs the existing workspace, which reads the source rasters, transforms them, and writes the resulting image into a local folder. Set "Yes" to the "Wait for Job to Complete" parameter of the WorkspaceRunner.
You can then use an S3Uploader connected to the Succeeded port of the WorkspaceRunner, to upload the resulting image after completing the translation.
Takashi
This is informative.
Just wanted to check if it is still the case of writing the output to the local disk and upload to s3 or there's been an upgrade ?
Thanks.
+50- Celebrity
raghunaren wrote:
This is informative.
Just wanted to check if it is still the case of writing the output to the local disk and upload to s3 or there's been an upgrade ?
Thanks.
Reply
Rich Text Editor, editor1
Editor toolbars
Press ALT 0 for help
Helpful Members This Week
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.