Skip to main content

Hi, I am using HTTPcaller in FME 2020 to send an HTTP request to a secured service. I have specified a basic authentication. When I run it, I get 401 error.

If I copy the cookie from Chrome dev tools to FME HTTPCaller header, it works.

However the cookie will expire, particularly as I will run my workspace once a week every week. How do I get a cookie generated for every call in HTTPcaller ?

I have tried Save Cookies option, but don't see any difference. The request fails when the cookie expires.

There is no single answer to this at it will depend on how the cookie is generated on the server side and what it is used for internally. Your best bet is probably to contact the server administrators and ask.


Hi @sunsilk11, as @david_r mentioned, the solution depends on how the server generates the cookies, but as a typical case, if there is a login page and it's simply implemented with a "form" element that posts the authentication parameters (e.g. username and password) when the user clicked [Login], it might be possible that you can use an HTTPCaller to emulate the login process (i.e. post the authentication parameters to the site through the login page) and save the cookies. If that was possible, you could access your desired page with a subsequent HTTPCaller.


Hi @sunsilk11, as @david_r mentioned, the solution depends on how the server generates the cookies, but as a typical case, if there is a login page and it's simply implemented with a "form" element that posts the authentication parameters (e.g. username and password) when the user clicked [Login], it might be possible that you can use an HTTPCaller to emulate the login process (i.e. post the authentication parameters to the site through the login page) and save the cookies. If that was possible, you could access your desired page with a subsequent HTTPCaller.

That's correct, there is a login form fot username/password. I was hoping I could get the cookie from a first request. By emulating login, do you mean using the login url in a httpcaller?


That's correct, there is a login form fot username/password. I was hoping I could get the cookie from a first request. By emulating login, do you mean using the login url in a httpcaller?

Yes, try this procedure.

Access the login URL to generate cookies with an HTTPCaller.

  • HTTP Method: GET
  • Save Cookies: Yes

With the second HTTPCaller, access the same login URL and post username and password.

  • HTTP Method: POST
  • Upload Data: Multipart / Form Data
  • Multipart Upload: <set required authentication parameters>
  • Save Cookies: Yes

Then, hopefully you can access your desired page with the third HTTPCaller (Save Cookies: Yes).

 

You will have to analyze the HTML document of the login page to know the required authentication parameters.

 


Hi @sunsilk11, as @david_r mentioned, the solution depends on how the server generates the cookies, but as a typical case, if there is a login page and it's simply implemented with a "form" element that posts the authentication parameters (e.g. username and password) when the user clicked [Login], it might be possible that you can use an HTTPCaller to emulate the login process (i.e. post the authentication parameters to the site through the login page) and save the cookies. If that was possible, you could access your desired page with a subsequent HTTPCaller.

@takashi

Thanks. I am just trying this suggestion.

Currently getting Could not verify the provided CSRF token because your session was not found in second HTTPcaller.

For Multipart Upload: <set required authentication parameters> ,I have tried username, password

Also tried Authentication Basic username:password (in dbase64)

Not sure how to get the session though via HTTPcaller. I was expecting this in the response from first httpcaller.

 

In Fiddler, I get following in the response when using same url as first httpcaller:

HTTP/1.1 200 OK

Server: Apache-Coyote/1.1

Access-Control-Allow-Origin: *

Access-Control-Allow-Methods: OPTIONS

Access-Control-Max-Age: 3600

Access-Control-Allow-Headers: x-requested-with, authorization

X-Content-Type-Options: nosniff

X-XSS-Protection: 1; mode=block

Cache-Control: no-cache, no-store, max-age=0, must-revalidate

Pragma: no-cache

Expires: 0

Strict-Transport-Security: max-age=31536000 ; includeSubDomains

X-Frame-Options: DENY

Set-Cookie: JSESSIONID=AAFCA6FA41B97C154B471DA16B45F5E9; Path=/; Secure; HttpOnly

Set-Cookie: XrefTOKEN=e6028831-3a27-4cb2-932c-068d0979dbff; Path=/

X-Application-Context: application:8443

Content-Type: text/html;charset=UTF-8

Content-Language: en-GB

Content-Length: 4800

Date: Tue, 23 Jun 2020 13:44:10 GMT

 

 


Reply