Solved

I'm trying to connect to a non Amazon hosted S3 Service. Is that possible using the S3Downloader Transformer

  • 2 March 2022
  • 4 replies
  • 27 views

Badge

Thinking other cloud providers provide S3 compatible storage. In this specific case we have an on prem S3 Storage device. I'd like to be able to pull the data from a bucket on that device. With most other tools I can override the hosts that the s3 protocol is trying to connect to. It doesn't look like that's the case for the fme s3 downloader, or am I missing something.

icon

Best answer by jlutherthomas 9 March 2022, 16:39

View original

4 replies

Userlevel 5
Badge +25

The S3Connector works with Amazon S3, if that other service is identical to S3 (which is not the same as compatible) it might work, but since you're having trouble I'm assuming it works differently in terms of authentication and/or api calls

Badge

Thanks for the answer.... It looks like S3 communication is via a rest api. I think other S3 Compatible providers just implement that api (example minio). There are a few tools out there that allow you to browse your s3 storage (S3-browser, cyberduck, etc). All those tools allow you to override default server/host values with the name of whatever host your s3 service is running on. It seemed odd that this wasn't the case with the various FME S3 transformers, thus the question, as I felt like maybe I was missing something...

Userlevel 5
Badge +29

It should be relatively straight forward to achieve the same outcomes using httpcallers and the services api documentation

Badge +2

@guy_lafleur​ 

I wrote this article about minIO and FME a while back, which worked as long as you change the endpoint.

https://community.safe.com/s/article/MinIO-and-FME

Reply