Question

Downloading a CSV from a password protected site

Forum|Forum|7 years ago
October 15, 2018
3 replies
101 views

clickingbuttons

I'm trying to download a file from a site that requires a password and a button press. I'm using HTTPCaller, with POST as method. The Body option is Multipart/Form Data, the Multipart Upload is

password, String Upload, (the password)

and

btn-open-file, String Upload, Open

Unfortunately, I'm only receiving the following error:

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>502 Proxy Error</title>
</head><body>
<h1>Proxy Error</h1>
<p>The proxy server received an invalid
response from an upstream server.<br />
The proxy server could not handle the request <em><a href="/public/file/1TcNS1xp_kyoFY4B3JiaUA/SSF-GIS-Daten.CSV">POST /public/file/1TcNS1xp_kyoFY4B3JiaUA/SSF-GIS-Daten.CSV</a></em>.<p>
Reason: <strong>Error reading from remote server</strong></p></p>
</body></html>

Also, this is the login form and source:

What's the issue here? I found this very similar problem but couldn't get anything from the discussion to work: https://knowledge.safe.com/questions/68018/download-zip-file-from-password-protected-website.html

This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

takashi
Forum|Forum|7 years ago
October 16, 2018

Hi @clickingbuttons, make sure that the request URL is set correctly. The request URL should point to the location specified by the "action" attribute in the "form" element. In this case, should look like this:

<the same directory as the web page>/public/file/somefile.CSV

Why not inspect features with Visual/Data Preview and Feature/Record Information before writing them into a destination dataset?

Upvote

takashi
Forum|Forum|7 years ago
October 16, 2018

Hi @clickingbuttons, make sure that the request URL is set correctly. The request URL should point to the location specified by the "action" attribute in the "form" element. In this case, should look like this:

<the same directory as the web page>/public/file/somefile.CSV

If the website requires login procedure before accessing the CSV file, the discuss in this thread might help you.