Skip to main content

Hi,

 

I am trying to download a csv with this url (https://waterinfo.rws.nl/api/Download/CSV?expertParameter=Waterhoogte___20Oppervlaktewater___20t.o.v.___20Normaal___20Amsterdams___20Peil___20in___20cm&locationSlug=Delfzijl(DLFZ)&timehorizon=-24,24) using FME.

The csv contains data about the water level in the past 24 hours and the expected water level in the next 24 hours for a specific location. I have tried with the http caller but that doensn't work. Is there a way to download the csv using FME or to read the data in another way with a FME reader/transformer? In the attachment I have included the csv that should be the result of the download. 

I hope someone can help me.

Peter

How do you setup your HTTPCaller? What HTTP error do you get? Can you provide a log?

Do you use GET, do you use POST, are you seting the Query String Parameters right?

Have you checked the API documentation?


How do you setup your HTTPCaller? What HTTP error do you get? Can you provide a log?

Do you use GET, do you use POST, are you seting the Query String Parameters right?

Have you checked the API documentation?

See for the HTTP Caller setup the attached simple workbench. I think the Query string parameters are right. The recieved error is : HTTPCaller(HTTPFactory): Received HTTP response header: 'HTTP/1.1 500 Internal Server Error' from 'https://waterinfo.rws.nl/api/Download/CSV?expertParameter=Waterhoogte___20Oppervlaktewater___20t.o.v.___20Normaal___20Amsterdams___20Peil___20in___20cm&locationSlug=Delfzijl%28DLFZ%29&timehorizon=-24%2C24'


I looked into the website. I can't figure out where the API documentation is located.

But I could figure out how to get the data in JSON.

Use this link https://waterinfo.rws.nl/api/chart?mapType=waterhoogte&locationCode=Delfzijl(DLFZ)&values=-48,48 to get a JSON response then Fragment all the data out of it.

I recommend using JSONFragmenter instead of HTTPCaller as it has a nice GUI for creating the JSON Query.


I looked into the website. I can't figure out where the API documentation is located.

But I could figure out how to get the data in JSON.

Use this link https://waterinfo.rws.nl/api/chart?mapType=waterhoogte&locationCode=Delfzijl(DLFZ)&values=-48,48 to get a JSON response then Fragment all the data out of it.

I recommend using JSONFragmenter instead of HTTPCaller as it has a nice GUI for creating the JSON Query.

Thanks! This helps! This alternative via JSON works fine, I was focused on using the HttpCaller, but this method is perfectly usable. Thanks for the support!

 


Hi @pschout​ 

It sounds like you already have a great answer from @caracadrian​  that works even better for your workflow but still wanted to share a tip in case it helps other users in the future.

If you are seeing a difference in behaviour between a web browser and HTTPCaller, it may be helpful to try emulating the browser's request in the HTTPCaller. In this case, it appears that setting the request headers to be the same as the web browser allows the GET request to succeed.

HTTPCaller_ParamYou can usually find this information via the Network tab after selecting Inspect Element.

WebBrowser_Network


Hi @pschout​ 

It sounds like you already have a great answer from @caracadrian​  that works even better for your workflow but still wanted to share a tip in case it helps other users in the future.

If you are seeing a difference in behaviour between a web browser and HTTPCaller, it may be helpful to try emulating the browser's request in the HTTPCaller. In this case, it appears that setting the request headers to be the same as the web browser allows the GET request to succeed.

HTTPCaller_ParamYou can usually find this information via the Network tab after selecting Inspect Element.

WebBrowser_Network

Hi debbiatsafe (Safer),

I tried your suggestion with the HTTPCaller and the Header information and it works fine, thanks for the tip!

 


Reply