Skip to main content
Question

HTTPCaller returning 403 error

  • August 25, 2023
  • 2 replies
  • 83 views

tnarladni
Enthusiast
Forum|alt.badge.img+23

This is the URL i'm trying to access.

https://qpublic.schneidercorp.com/Application.aspx?AppID=1048&PageTypeID=4&KeyValue=450070160000

It's a public link. If I try to access it via the HTTPCaller, it throws a 403 error. Why would accessing the link via HTTPCaller throws an error? Thanks

2 replies

redgeographics
Celebrity
Forum|alt.badge.img+59
  • Celebrity
  • 3700 replies
  • August 26, 2023

Maybe it's blocked based on the user-agent, to prevent scraping the site?


thijsknapen
Contributor
Forum|alt.badge.img+11
  • Contributor
  • 155 replies
  • August 26, 2023

Well, the website seems to perform some kind of verification step (e.g. try it out in the incognito mode of your browser).

 

If I follow the traffic in the browser, I notice that also there the initial response is 403, but after some additional steps/requests (the verification steps), the result is returned.

I suppose if you would be able to mimic the verification steps in FME, you could make it work, but I think that may be quite difficult (I'm not able to help you much further with that).

 

Maybe you can use a PythonCaller that makes use of a Selenium module (?) for this. See also; Selenium

Selenium can be used to automate browser actions (I have mainly heard of it being used for testing (web) applications).