Skip to main content
Solved

How to add a custom certificate to a HTTPS request ?


lifalin2016
Contributor
Forum|alt.badge.img+29

Hi.

I'm exploring the possibility of making request to a secure server using FME.

The webservices to be called require that the requests are accompanied by a company or function certificate. It will typically be stored in a certificate repository on the machine.

I can make it work manually using HttpCaller with authentication = "Single Sign-On", but this prompts me for a certificate password every time, and the final script needs to run fully autonomously (on an FME Server).

Any help is appreciated.

Cheers.

Best answer by lifalin2016

So, after a long period of testing, I managed to get something working with a PythonCaller.

After debugging and testing the code in PyCharms, I back ported the code to FME, and found that 2018.1 wasn't up for the task. It required Python 3.7, so I switched to to 2020.0, which do support 3.7.

I need the socket and ssl modules for the task, and created a secure connection with my personal employee (MOCES/X.509) certificate. It required that the certificate was split into two files: one with the public key and one with the private key. And the password for the files. If anyone's facing a similar task, I can share my raw code.

Unfortunately this still doesn't enable HttpCaller usage, as this isn't capable of creating a secure connection with a certificate. And sharing a secure connection across multiple transformers doesn't seem possible either, but at least one can share the response data.

I consider this matter closed, this being the conclusive answer for now.

View original
Did this help you find an answer to your question?

12 replies

jovitaatsafe
Safer
Forum|alt.badge.img+11

Hi @lifalin2016,

I'm not well versed in this topic, but would this Q&A provide any leads on what you're trying to do? There's a bit more on the parameters of the HTTPCaller under Verify SSL Certificates. If that's not it, perhaps someone else from the Community will be able to chime in with some other ideas?


lifalin2016
Contributor
Forum|alt.badge.img+29
  • Author
  • Contributor
  • February 18, 2020
jovitaatsafe wrote:

Hi @lifalin2016,

I'm not well versed in this topic, but would this Q&A provide any leads on what you're trying to do? There's a bit more on the parameters of the HTTPCaller under Verify SSL Certificates. If that's not it, perhaps someone else from the Community will be able to chime in with some other ideas?

Unfortunately it has nothing to do with SSL, but with function certificates (FOCES).

Anyone?


Forum|alt.badge.img+2
lifalin2016 wrote:

Unfortunately it has nothing to do with SSL, but with function certificates (FOCES).

Anyone?

@lifalin2016 Do you have any information on FOCES certificates? I couldn't find anything online as to how they might work with a webservice.

 

 

Where is the certificate located that you're providing a password for? Do you have a screenshot of how it's prompting you?

 

 

Are you able to get the HTTPCaller to work without prompting?

lifalin2016
Contributor
Forum|alt.badge.img+29
  • Author
  • Contributor
  • February 21, 2020

Ah, it looks like the "OCES" (FOCES = Function OCES) is a Danish denomination :-/

In reality it's a plain X.509 certificate as I understand it, used to sign requests (apart from the encrypted transport like SSL). If no certificate accompanies the initial request, a request for an authenticated request is returned by the server, and this is where the popup dialog comes up on my client. I want to skip that.

So my question is how to attach such an X.509 certificate to the initial HttpRequest to avoid this interactive back'n'forth. I have the certificate stored in a PKCS12 formatted file.

I only have two options in HttpCaller - headers and attachments. Which can I use ?

I know how to do it in C#, but I would very much like to hear about what's "under the hood" from someone at Safe, please?


Forum|alt.badge.img+2
lifalin2016 wrote:

Ah, it looks like the "OCES" (FOCES = Function OCES) is a Danish denomination :-/

In reality it's a plain X.509 certificate as I understand it, used to sign requests (apart from the encrypted transport like SSL). If no certificate accompanies the initial request, a request for an authenticated request is returned by the server, and this is where the popup dialog comes up on my client. I want to skip that.

So my question is how to attach such an X.509 certificate to the initial HttpRequest to avoid this interactive back'n'forth. I have the certificate stored in a PKCS12 formatted file.

I only have two options in HttpCaller - headers and attachments. Which can I use ?

I know how to do it in C#, but I would very much like to hear about what's "under the hood" from someone at Safe, please?

Hi @lifalin2016,

 

Without knowing how you would successfully do this in C# (or postman/cURL) (if you do, this fresh new question posting might be of use: https://knowledge.safe.com/questions/108492/question-of-the-week-web-services-curl-and-the-htt.html)

 

 

Otherwise my best guess would be to send the certificate using the 'Upload Data' set to either 'Multipart/Form Data' or 'Upload From File'.

 

 

If those don't work, if you could share how you do it in C# that might help us be able to work out what you be the best method of doing it in the HTTPCaller.

lifalin2016
Contributor
Forum|alt.badge.img+29
  • Author
  • Contributor
  • February 25, 2020
jlutherthomas wrote:

Hi @lifalin2016,

 

Without knowing how you would successfully do this in C# (or postman/cURL) (if you do, this fresh new question posting might be of use: https://knowledge.safe.com/questions/108492/question-of-the-week-web-services-curl-and-the-htt.html)

 

 

Otherwise my best guess would be to send the certificate using the 'Upload Data' set to either 'Multipart/Form Data' or 'Upload From File'.

 

 

If those don't work, if you could share how you do it in C# that might help us be able to work out what you be the best method of doing it in the HTTPCaller.

I'm not entirely sure what you mean by "Upload Data" or "Upload From File". I don't see such options in the HttpCaller interface !??


lifalin2016
Contributor
Forum|alt.badge.img+29
  • Author
  • Contributor
  • February 26, 2020
lifalin2016 wrote:

I'm not entirely sure what you mean by "Upload Data" or "Upload From File". I don't see such options in the HttpCaller interface !??

Ah, they appear when you switch from GET to POST. Unfortunately this is not possible in this case.


lifalin2016
Contributor
Forum|alt.badge.img+29
  • Author
  • Contributor
  • February 26, 2020

After some digging around, it seems that the client certificate needs to be invoked when the connection is established, not for each request. This makes sense, I guess.

Unfortunately HttpCaller does not have any provisions for adding certificates to the connection to be established. And I'm a little unsure whether such a certified tunnel is reused across multiple HttpCaller's ?

I did read "Question of the week..." but I don't think it adresses my situation, only how to fetch tokens in token based services with url based parameters (and they're not hard to do).

I've looked into Python examples of creating secure connections, but the problem is of course keeping the context, so requests are made in that secure context across several transformers.

Does anyone have some insights into secure tunnels with FME ?


lifalin2016
Contributor
Forum|alt.badge.img+29
  • Author
  • Contributor
  • Best Answer
  • March 13, 2020

So, after a long period of testing, I managed to get something working with a PythonCaller.

After debugging and testing the code in PyCharms, I back ported the code to FME, and found that 2018.1 wasn't up for the task. It required Python 3.7, so I switched to to 2020.0, which do support 3.7.

I need the socket and ssl modules for the task, and created a secure connection with my personal employee (MOCES/X.509) certificate. It required that the certificate was split into two files: one with the public key and one with the private key. And the password for the files. If anyone's facing a similar task, I can share my raw code.

Unfortunately this still doesn't enable HttpCaller usage, as this isn't capable of creating a secure connection with a certificate. And sharing a secure connection across multiple transformers doesn't seem possible either, but at least one can share the response data.

I consider this matter closed, this being the conclusive answer for now.


oliver.morris
Contributor
Forum|alt.badge.img+12

I also have the same issue, it would be good to extend HTTPCaller to cover this


@Lars I Nielsen​ I have the same issue. I was trying to get it work with the python caller. Unfortunately it does not work. It is possible to share the Python code?


lifalin2016
Contributor
Forum|alt.badge.img+29
  • Author
  • Contributor
  • September 9, 2022
dj_huisman wrote:

@Lars I Nielsen​ I have the same issue. I was trying to get it work with the python caller. Unfortunately it does not work. It is possible to share the Python code? 

Here's the core code of my trials. Substitute your own arguments in the relevant calls. You need seperate certificate and key files.

import socket
import ssl
 
context = ssl.create_default_context(ssl.Purpose.SERVER_AUTH)
# context = ssl.SSLContext(ssl.PROTOCOL_SSLv3)
 
context.load_cert_chain(certfile=client_cert, keyfile=client_keys, password=client_pass)
 
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
conn = context.wrap_socket(s, server_side=False, server_hostname=host_name)
 
conn.connect((host_name, host_port))
 
# and then
req = "GET /api/{1} HTTP/1.1\r\nHost: {0}".format(host_name, test_page, host_port)
conn.send((req + "\r\n\r\n ").encode())  # send as bytes

Good luck. Cheers.


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings