Solved

BC Data Catalogue

  • 16 January 2019
  • 2 replies
  • 11 views
dmerrick
Rank
Badge

Hello,

Does anyone have experience automating the download of data packages from the BC Data Catalogue (https://catalogue.data.gov.bc.ca/dataset?download_audience=Public)?

 

There are dozens of provincial datasets I would like to automate the download of on a weekly basis to keep the data updated for my organization, the problem is that they require submitting a AOI (or mapsheet) and an email before receiving the zipped data package in an email.

Thanks!

 

icon

Best answer by virtualcitymatt 18 January 2019, 16:48

View original

2 replies

virtualcitymatt
Rank
Userlevel 4
Badge +26

I would contact them directly. They may have a API you can use to gain access and help you to build requests.

 

 

If you open the developer mode of your browser window just before you make you submit your order, you will see how you request is structured, where the request is going and what type of request it is (POST, GET, etc).

 

 

Indeed it looks like they use an FME Server to process the data and deliver you the download link via email.

Attached is the structure of a request which I made to a random data layer. You may be able to use this method as a guide to submit your own requests but I would still recommend contacting them directly, trying to build your own queries may get you blocked.

 

 

But ff you do want to try and build your own requests then I would suggest start working with a free tool called 'Postman' once you are able to get a good working response then you can put it into FME into a workspace with the HTTPCaller.

 

dmerrick
Rank
Badge

I would contact them directly. They may have a API you can use to gain access and help you to build requests.

 

 

If you open the developer mode of your browser window just before you make you submit your order, you will see how you request is structured, where the request is going and what type of request it is (POST, GET, etc).

 

 

Indeed it looks like they use an FME Server to process the data and deliver you the download link via email.

Attached is the structure of a request which I made to a random data layer. You may be able to use this method as a guide to submit your own requests but I would still recommend contacting them directly, trying to build your own queries may get you blocked.

 

 

But ff you do want to try and build your own requests then I would suggest start working with a free tool called 'Postman' once you are able to get a good working response then you can put it into FME into a workspace with the HTTPCaller.

 

Thanks for the response, I have contacted them directly, I posted on the forum to cover all possible options. I do not have experience with developer mode in the browser but will explore this option.

Reply


Community Stats

28,666
Posts
109,342
Replies
36,535
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.