+1Hello,
Does anyone have experience automating the download of data packages from the BC Data Catalogue (https://catalogue.data.gov.bc.ca/dataset?download_audience=Public)?
There are dozens of provincial datasets I would like to automate the download of on a weekly basis to keep the data updated for my organization, the problem is that they require submitting a AOI (or mapsheet) and an email before receiving the zipped data package in an email.
Thanks!
Best answer by virtualcitymatt
I would contact them directly. They may have a API you can use to gain access and help you to build requests.
If you open the developer mode of your browser window just before you make you submit your order, you will see how you request is structured, where the request is going and what type of request it is (POST, GET, etc).
Indeed it looks like they use an FME Server to process the data and deliver you the download link via email.
Attached is the structure of a request which I made to a random data layer. You may be able to use this method as a guide to submit your own requests but I would still recommend contacting them directly, trying to build your own queries may get you blocked.
But ff you do want to try and build your own requests then I would suggest start working with a free tool called 'Postman' once you are able to get a good working response then you can put it into FME into a workspace with the HTTPCaller.
+47I would contact them directly. They may have a API you can use to gain access and help you to build requests.
If you open the developer mode of your browser window just before you make you submit your order, you will see how you request is structured, where the request is going and what type of request it is (POST, GET, etc).
Indeed it looks like they use an FME Server to process the data and deliver you the download link via email.
Attached is the structure of a request which I made to a random data layer. You may be able to use this method as a guide to submit your own requests but I would still recommend contacting them directly, trying to build your own queries may get you blocked.
But ff you do want to try and build your own requests then I would suggest start working with a free tool called 'Postman' once you are able to get a good working response then you can put it into FME into a workspace with the HTTPCaller.
+1I would contact them directly. They may have a API you can use to gain access and help you to build requests.
If you open the developer mode of your browser window just before you make you submit your order, you will see how you request is structured, where the request is going and what type of request it is (POST, GET, etc).
Indeed it looks like they use an FME Server to process the data and deliver you the download link via email.
Attached is the structure of a request which I made to a random data layer. You may be able to use this method as a guide to submit your own requests but I would still recommend contacting them directly, trying to build your own queries may get you blocked.
But ff you do want to try and build your own requests then I would suggest start working with a free tool called 'Postman' once you are able to get a good working response then you can put it into FME into a workspace with the HTTPCaller.
Thanks for the response, I have contacted them directly, I posted on the forum to cover all possible options. I do not have experience with developer mode in the browser but will explore this option.
