Solved

How to add a custom certificate to a HTTPS request ?

  • 7 February 2020
  • 12 replies
  • 112 views

Userlevel 1
Badge +22

Hi.

I'm exploring the possibility of making request to a secure server using FME.

The webservices to be called require that the requests are accompanied by a company or function certificate. It will typically be stored in a certificate repository on the machine.

I can make it work manually using HttpCaller with authentication = "Single Sign-On", but this prompts me for a certificate password every time, and the final script needs to run fully autonomously (on an FME Server).

Any help is appreciated.

Cheers.

icon

Best answer by lifalin2016 13 March 2020, 10:21

View original

12 replies

Badge +11

Hi @lifalin2016,

I'm not well versed in this topic, but would this Q&A provide any leads on what you're trying to do? There's a bit more on the parameters of the HTTPCaller under Verify SSL Certificates. If that's not it, perhaps someone else from the Community will be able to chime in with some other ideas?

Userlevel 1
Badge +22

Hi @lifalin2016,

I'm not well versed in this topic, but would this Q&A provide any leads on what you're trying to do? There's a bit more on the parameters of the HTTPCaller under Verify SSL Certificates. If that's not it, perhaps someone else from the Community will be able to chime in with some other ideas?

Unfortunately it has nothing to do with SSL, but with function certificates (FOCES).

Anyone?

Badge +2

Unfortunately it has nothing to do with SSL, but with function certificates (FOCES).

Anyone?

@lifalin2016 Do you have any information on FOCES certificates? I couldn't find anything online as to how they might work with a webservice.

 

 

Where is the certificate located that you're providing a password for? Do you have a screenshot of how it's prompting you?

 

 

Are you able to get the HTTPCaller to work without prompting?
Userlevel 1
Badge +22

Ah, it looks like the "OCES" (FOCES = Function OCES) is a Danish denomination :-/

In reality it's a plain X.509 certificate as I understand it, used to sign requests (apart from the encrypted transport like SSL). If no certificate accompanies the initial request, a request for an authenticated request is returned by the server, and this is where the popup dialog comes up on my client. I want to skip that.

So my question is how to attach such an X.509 certificate to the initial HttpRequest to avoid this interactive back'n'forth. I have the certificate stored in a PKCS12 formatted file.

I only have two options in HttpCaller - headers and attachments. Which can I use ?

I know how to do it in C#, but I would very much like to hear about what's "under the hood" from someone at Safe, please?

Badge +2

Ah, it looks like the "OCES" (FOCES = Function OCES) is a Danish denomination :-/

In reality it's a plain X.509 certificate as I understand it, used to sign requests (apart from the encrypted transport like SSL). If no certificate accompanies the initial request, a request for an authenticated request is returned by the server, and this is where the popup dialog comes up on my client. I want to skip that.

So my question is how to attach such an X.509 certificate to the initial HttpRequest to avoid this interactive back'n'forth. I have the certificate stored in a PKCS12 formatted file.

I only have two options in HttpCaller - headers and attachments. Which can I use ?

I know how to do it in C#, but I would very much like to hear about what's "under the hood" from someone at Safe, please?

Hi @lifalin2016,

 

Without knowing how you would successfully do this in C# (or postman/cURL) (if you do, this fresh new question posting might be of use: https://knowledge.safe.com/questions/108492/question-of-the-week-web-services-curl-and-the-htt.html)

 

 

Otherwise my best guess would be to send the certificate using the 'Upload Data' set to either 'Multipart/Form Data' or 'Upload From File'.

 

 

If those don't work, if you could share how you do it in C# that might help us be able to work out what you be the best method of doing it in the HTTPCaller.
Userlevel 1
Badge +22

Hi @lifalin2016,

 

Without knowing how you would successfully do this in C# (or postman/cURL) (if you do, this fresh new question posting might be of use: https://knowledge.safe.com/questions/108492/question-of-the-week-web-services-curl-and-the-htt.html)

 

 

Otherwise my best guess would be to send the certificate using the 'Upload Data' set to either 'Multipart/Form Data' or 'Upload From File'.

 

 

If those don't work, if you could share how you do it in C# that might help us be able to work out what you be the best method of doing it in the HTTPCaller.

I'm not entirely sure what you mean by "Upload Data" or "Upload From File". I don't see such options in the HttpCaller interface !??

Userlevel 1
Badge +22

I'm not entirely sure what you mean by "Upload Data" or "Upload From File". I don't see such options in the HttpCaller interface !??

Ah, they appear when you switch from GET to POST. Unfortunately this is not possible in this case.

Userlevel 1
Badge +22

After some digging around, it seems that the client certificate needs to be invoked when the connection is established, not for each request. This makes sense, I guess.

Unfortunately HttpCaller does not have any provisions for adding certificates to the connection to be established. And I'm a little unsure whether such a certified tunnel is reused across multiple HttpCaller's ?

I did read "Question of the week..." but I don't think it adresses my situation, only how to fetch tokens in token based services with url based parameters (and they're not hard to do).

I've looked into Python examples of creating secure connections, but the problem is of course keeping the context, so requests are made in that secure context across several transformers.

Does anyone have some insights into secure tunnels with FME ?

Userlevel 1
Badge +22

So, after a long period of testing, I managed to get something working with a PythonCaller.

After debugging and testing the code in PyCharms, I back ported the code to FME, and found that 2018.1 wasn't up for the task. It required Python 3.7, so I switched to to 2020.0, which do support 3.7.

I need the socket and ssl modules for the task, and created a secure connection with my personal employee (MOCES/X.509) certificate. It required that the certificate was split into two files: one with the public key and one with the private key. And the password for the files. If anyone's facing a similar task, I can share my raw code.

Unfortunately this still doesn't enable HttpCaller usage, as this isn't capable of creating a secure connection with a certificate. And sharing a secure connection across multiple transformers doesn't seem possible either, but at least one can share the response data.

I consider this matter closed, this being the conclusive answer for now.

Badge +10

I also have the same issue, it would be good to extend HTTPCaller to cover this

@Lars I Nielsen​ I have the same issue. I was trying to get it work with the python caller. Unfortunately it does not work. It is possible to share the Python code?

Userlevel 1
Badge +22

@Lars I Nielsen​ I have the same issue. I was trying to get it work with the python caller. Unfortunately it does not work. It is possible to share the Python code? 

Here's the core code of my trials. Substitute your own arguments in the relevant calls. You need seperate certificate and key files.

import socket
import ssl
 
context = ssl.create_default_context(ssl.Purpose.SERVER_AUTH)
# context = ssl.SSLContext(ssl.PROTOCOL_SSLv3)
 
context.load_cert_chain(certfile=client_cert, keyfile=client_keys, password=client_pass)
 
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
conn = context.wrap_socket(s, server_side=False, server_hostname=host_name)
 
conn.connect((host_name, host_port))
 
# and then
req = "GET /api/{1} HTTP/1.1\r\nHost: {0}".format(host_name, test_page, host_port)
conn.send((req + "\r\n\r\n ").encode())  # send as bytes

Good luck. Cheers.

Reply