I'm attempting to connect to an AWS S3 bucket from FME desktop (2018.0). (Colleagues have informed me that the situation is no different in FME 2019). The system proxy settings are controlled via a PAC file as part of group policy. I know that the proxy is NTLM, and as far as I can tell there is no way that FME can pass on authentication credentials correctly and so connections are always refused. I have tried the various settings within tools>FMEoptions>Network Proxy> but I always receive the same connection error -'Unable to execute HTTP request: '
I can navigate to the bucket via the browser with no problems.
Is the only alternative (in my case to pay for) a proxy authentication bypass?
Best answer by virtualcitymatt
I *believe* that FME transformers which are python based do not support the NTLM authentication and I suspect that the s3 connectors are in fact pyhton based. This looks like it's supported by the article you've linked.
Can you try perhaps building the Request with an HTTPCaller? If you are simple trying to get a file from the bucket then the HTTP Caller *should* work with NTLM. A good was to track what steps FMEs webtools are doing is often to turn on the DEBUG log in FME (Just make sure to ignore all the BADNEWS stuff). With the HTTPCaller you should see a little more information about FME Trying to connect through the proxy or not.
I *believe* that FME transformers which are python based do not support the NTLM authentication and I suspect that the s3 connectors are in fact pyhton based. This looks like it's supported by the article you've linked.
Can you try perhaps building the Request with an HTTPCaller? If you are simple trying to get a file from the bucket then the HTTP Caller *should* work with NTLM. A good was to track what steps FMEs webtools are doing is often to turn on the DEBUG log in FME (Just make sure to ignore all the BADNEWS stuff). With the HTTPCaller you should see a little more information about FME Trying to connect through the proxy or not.
I *believe* that FME transformers which are python based do not support the NTLM authentication and I suspect that the s3 connectors are in fact pyhton based. This looks like it's supported by the article you've linked.
Can you try perhaps building the Request with an HTTPCaller? If you are simple trying to get a file from the bucket then the HTTP Caller *should* work with NTLM. A good was to track what steps FMEs webtools are doing is often to turn on the DEBUG log in FME (Just make sure to ignore all the BADNEWS stuff). With the HTTPCaller you should see a little more information about FME Trying to connect through the proxy or not.
could you explain which headers would be required for an http put request to an aws s3 bucket? I seem to be generating a lot of http 400 bad requests... Is it possibleto force https for these requests? Sorry - sounds like I need some s3 training!
could you explain which headers would be required for an http put request to an aws s3 bucket? I seem to be generating a lot of http 400 bad requests... Is it possibleto force https for these requests? Sorry - sounds like I need some s3 training!
I'm afraid that I'm also not really the export here, if you want to experience with REST Calls, I would recommend that you use PostMan. It's a pretty handy tool which you can work with to build your requests. Once you have something working you can then try with the HTTPCaller
It's worth checking. Otherwise perhaps IT have a different method which you could use for the proxy? or whitelist it so that requests to S3 skip the proxy?
I *believe* that FME transformers which are python based do not support the NTLM authentication and I suspect that the s3 connectors are in fact pyhton based. This looks like it's supported by the article you've linked.
Can you try perhaps building the Request with an HTTPCaller? If you are simple trying to get a file from the bucket then the HTTP Caller *should* work with NTLM. A good was to track what steps FMEs webtools are doing is often to turn on the DEBUG log in FME (Just make sure to ignore all the BADNEWS stuff). With the HTTPCaller you should see a little more information about FME Trying to connect through the proxy or not.
Thanks MAtt, A quick look tells me that for me learning the http, authentication requirements and relevant interaction at the low level is likely to be less cost effective than paying for a proxy authentication - so I will probably go down that route.
Thanks MAtt, A quick look tells me that for me learning the http, authentication requirements and relevant interaction at the low level is likely to be less cost effective than paying for a proxy authentication - so I will probably go down that route.
Fair enough, i's especially complicated when working with AWS as well. *headache emoji*. Good luck with your work
We use 3 different kinds of cookies. You can choose which cookies you want to accept. We need basic cookies to make this site work, therefore these are the minimum you can select. Learn more about our cookies.
