Skip to main content

This is the URL i'm trying to access.

https://qpublic.schneidercorp.com/Application.aspx?AppID=1048&PageTypeID=4&KeyValue=450070160000

It's a public link. If I try to access it via the HTTPCaller, it throws a 403 error. Why would accessing the link via HTTPCaller throws an error? Thanks

Maybe it's blocked based on the user-agent, to prevent scraping the site?


Well, the website seems to perform some kind of verification step (e.g. try it out in the incognito mode of your browser).

 

If I follow the traffic in the browser, I notice that also there the initial response is 403, but after some additional steps/requests (the verification steps), the result is returned.

I suppose if you would be able to mimic the verification steps in FME, you could make it work, but I think that may be quite difficult (I'm not able to help you much further with that).

 

Maybe you can use a PythonCaller that makes use of a Selenium module (?) for this. See also; Selenium

Selenium can be used to automate browser actions (I have mainly heard of it being used for testing (web) applications).

 


Reply