Question

HTTP/FTP transfer error: 'SSL connect error'


Hello,

 

I want to access an api (not open to public but from one of our suppliers) using the HttpCaller .

Connecting to the API with postman works fine.

 

Connecting from the same server with FME gives the error:

 

HTTPCaller (HTTPFactory): HTTP/FTP transfer error: 'SSL connect error'

HTTPCaller (HTTPFactory): Please ensure that your network connection is properly set up

HTTPCaller (HTTPFactory): No proxy settings have been entered. If you require a proxy to access external URLs, please ensure the appropriate information has been entered

 

Connecting with FME from another environment works fine.

 

We dont use a proxy.

I have changed the option: Verify SSL certificates to NO

 

Any ideas on what could be the problem?


13 replies

Userlevel 4
Badge +26

You could try temporarily turning on debug logging that might give you some more clues as to what's going on. I've found debug logging quite helpful for the HTTPCaller.​

Thank you for this answer and sorry i wasnt able to respond. Dont work on the weekends ;)

 

I enabled debog mode and it somewhat gave me the following information:

 

HTTPCaller (HTTPFactory): HTTP info: Connected to mijn.schouwen-duiveland.nl (99.83.202.219) port 443 (#0)

HTTPCaller (HTTPFactory): HTTP info: schannel: next InitializeSecurityContext failed: SEC_E_ILLEGAL_MESSAGE (0x80090326) - This error usually occurs when a fatal SSL/TLS alert is received (e.g. handshake failed).

HTTPCaller (HTTPFactory): HTTP info: Closing connection 0

HTTPCaller (HTTPFactory): HTTP info: schannel: shutting down SSL/TLS connection with mijn.schouwen-duiveland.nl port 443

 

Looking up this error, i found the following topic on stackoverflow (not an FME topic but still the same error)

 

[Update 2020-10-17]

This solution has also been reported on the weathercan issue

It looks like curl uses two SSL backends for Windows: OpenSSL and Windows Secure Channel. By default curl uses the Windows Secure Channel which is supposed to be best, but doesn't quite have the full functionality of OpenSSL (more details).

My best guess is that something changed on the ECCC server that required a functionality not provided by Windows Secure Channel. When I forced my windows testing platform (AppVeyor) to use OpenSSL I had no more problems.

If you would like to try this, you'll have to add the following to your .Renviron file (which is loaded during R's startup):

CURL_SSL_BACKEND=openssl

 

I guess HTTPCaller uses Curl to connect to the API?

Any idea if there is a way for me to change this SSL option in FME?

 

 

Userlevel 4
Badge +26

Thank you for this answer and sorry i wasnt able to respond. Dont work on the weekends ;)

 

I enabled debog mode and it somewhat gave me the following information:

 

HTTPCaller (HTTPFactory): HTTP info: Connected to mijn.schouwen-duiveland.nl (99.83.202.219) port 443 (#0)

HTTPCaller (HTTPFactory): HTTP info: schannel: next InitializeSecurityContext failed: SEC_E_ILLEGAL_MESSAGE (0x80090326) - This error usually occurs when a fatal SSL/TLS alert is received (e.g. handshake failed).

HTTPCaller (HTTPFactory): HTTP info: Closing connection 0

HTTPCaller (HTTPFactory): HTTP info: schannel: shutting down SSL/TLS connection with mijn.schouwen-duiveland.nl port 443

 

Looking up this error, i found the following topic on stackoverflow (not an FME topic but still the same error)

 

[Update 2020-10-17]

This solution has also been reported on the weathercan issue

It looks like curl uses two SSL backends for Windows: OpenSSL and Windows Secure Channel. By default curl uses the Windows Secure Channel which is supposed to be best, but doesn't quite have the full functionality of OpenSSL (more details).

My best guess is that something changed on the ECCC server that required a functionality not provided by Windows Secure Channel. When I forced my windows testing platform (AppVeyor) to use OpenSSL I had no more problems.

If you would like to try this, you'll have to add the following to your .Renviron file (which is loaded during R's startup):

CURL_SSL_BACKEND=openssl

 

I guess HTTPCaller uses Curl to connect to the API?

Any idea if there is a way for me to change this SSL option in FME?

 

 

yes it does use curl in the backend.

 

You could try setting the environment variable as mentioned: CURL_SSL_BACKEND=openssl

It's listed here in the curl documentation: https://curl.se/libcurl/c/libcurl-env.html

 

I couldn't tell you whether or not this will actually work and maaay break other functionality in FME. but it's definitely worth a test.

 

Perhaps @nampreetatsafe​ can ask the Safe dev team about this?

Ok, so i'm having a bit of trouble understanding how to set these EV.

Could you point me in de right direction?

 

I first thought it was under windows EV, but i dont see a CURL installation (though Curl -help says Curl is installed)

 

Secondly i thought it would be in the HTTPCaller under Query string parameters, but then i get a call on: 'https://mijn.schouwen-duiveland.nl/api/v1/casetype?CURL_SSL_BACKEND=Openssl'

 

 

Userlevel 4
Badge +26

Ok, so i'm having a bit of trouble understanding how to set these EV.

Could you point me in de right direction?

 

I first thought it was under windows EV, but i dont see a CURL installation (though Curl -help says Curl is installed)

 

Secondly i thought it would be in the HTTPCaller under Query string parameters, but then i get a call on: 'https://mijn.schouwen-duiveland.nl/api/v1/casetype?CURL_SSL_BACKEND=Openssl'

 

 

Yeah it's just a windows environment variable you need to set. Add a new one called CURL_SSL_BACKEND and set the value to openssl

 

Start> search for "environment variables" > Select Edit System Environment Variables > choose "Environment Variables..." button on bottom right of window > Add a new variable in the system section .

 

You should probably rastet FME after setting it. You don't need to include it on the HTTP call.

Ok.. well that seems to do nothing.

I set it up;

 

Knipsel3I closed FME, rebooted the whole pc and still got;

 

HTTPCaller (HTTPFactory): HTTP info: schannel: next InitializeSecurityContext failed: SEC_E_ILLEGAL_MESSAGE (0x80090326) - This error usually occurs when a fatal SSL/TLS alert is received (e.g. handshake failed).

 

I read somewhere:

 

Go to http://curl.haxx.se/docs/caextract.html and download the digital certificate file named cacert.pem.

The PEM file contains a bundle of valid digital certificates. The certificates are used to verify the authenticity of secure websites. They're distributed by certificate authority (CA) companies such as GlobalSign and VeriSign.

 

Move the cacert.pem file to your C:\\curl folder and rename it curl-ca-bundle.crt.

 

Any idea if this could be it and if so, where should i place these when FME uses curllib.

 

Userlevel 4
Badge +26

Ok.. well that seems to do nothing.

I set it up;

 

Knipsel3I closed FME, rebooted the whole pc and still got;

 

HTTPCaller (HTTPFactory): HTTP info: schannel: next InitializeSecurityContext failed: SEC_E_ILLEGAL_MESSAGE (0x80090326) - This error usually occurs when a fatal SSL/TLS alert is received (e.g. handshake failed).

 

I read somewhere:

 

Go to http://curl.haxx.se/docs/caextract.html and download the digital certificate file named cacert.pem.

The PEM file contains a bundle of valid digital certificates. The certificates are used to verify the authenticity of secure websites. They're distributed by certificate authority (CA) companies such as GlobalSign and VeriSign.

 

Move the cacert.pem file to your C:\\curl folder and rename it curl-ca-bundle.crt.

 

Any idea if this could be it and if so, where should i place these when FME uses curllib.

 

'Openssl' It should not have a capital 'O' try it all lowercase.

 

I'm not sure about your follow up question, sorry. But I suspect that FME has that already integrated.

Ah, yes sorry about that. I first tried it with lower-case and after that upper-case.

Doesnt matter what i try, still get the error.

 

The documentation states:

 

SSL backend names (case-insensitive): bearssl, gnutls, gskit, mbedtls, mesalink, nss, openssl, rustls, schannel, secure-transport, wolfssl

Another option i'll look into, will report back later;

 

https://caddy.community/t/why-curl-fails-while-browser-calls-succeed/9841/4

I think you’ll want to use the --resolve option of curl for this. The reason its failing is because setting the Host header isn’t enough to have SNI be set for the TLS handshake

Userlevel 4
Badge +26

Ah, yes sorry about that. I first tried it with lower-case and after that upper-case.

Doesnt matter what i try, still get the error.

 

The documentation states:

 

SSL backend names (case-insensitive): bearssl, gnutls, gskit, mbedtls, mesalink, nss, openssl, rustls, schannel, secure-transport, wolfssl

well spotted. That's too bad. Sorry i'm all out of ideas here. You might need to get in touch with safe support on this matter

Ok so a little update; I used fiddler to check how Postman is connecting and how FME is connecting. I see different Cipher Suites beeing used. Dont know how that is possible and how i can add ciphersuites to FME, but maybe others might have an idea?

 

If someone other then me has this problem and want to try out Fiddler. First i had to make sure FME traffic was routed to Fiddler (else it wouldnt show up in the log). This is done by setting a proxy to the same machine. See: https://community.safe.com/s/question/0D54Q000080hfSN/view-calls-from-httpcaller-in-fiddler-over-ssl

 

 

Badge +10

Hi @nickblakenburg​ 

 

That a good find. Postman uses the cypher of the web browser, while FME uses the cyphers installed on the operating system. We don't ship or support the ciphers. Most likely they will need to use newer operating system to allow the connection. This link has details on what cyphers are included with various Windows operating systems.

 

Additionally it looks like @virtualcitymatt​ environmental variable method should work but unfortunately only on builds 2021.1 and newer builds

 

 

Perfect! I updated my version to 2021.1 and added the windows environment variable CURL_SSL_BACKEND and set the value to openssl.

 

Now it works!

Reply