I have successfully used readers that point to download links for datasets in the past. I have a situation I am up against now where using this method does not work. I have two different links I am trying to read from:
This link takes you to a map viewer with a button that allows you to download either a geopackage or a shapefile of the dataset. However, when I try to right click on the download button, there is not an option to copy the URL of the button itself, so I am not able to use a reader to download then process the data.
When I click this link it downloads a zipped geopackage without taking me to a web page. Similarly, I cannot get a reader to successfully read this link.
Does anyone have any advice/tips/solutions for either of the above two items? Thank you in advance!
Best answer by virtualcitymatt
For Link one it looks like the data is being compiled based on a request. As a tip you can open up the develper tap in your browser and look at the network traffic when you hit the butten. You will likely see some kind of query. (URL + some settings) The settings are set based on user the user input like published parameters in an FME workspace so the the download url is not static
Here's a screen shot of what I mean from looking at link 1
Here at the top or the right window you can see the base URL. At the bottom are the request parameters. If you switch the view from "From Data" to "View Source" you can see the end part of the URL
There is another method. Just because we have the URL here unfortunately when putting it in a Reader FME removes all the stuff off the end and adds a .gpkg extension to the URL.
What you want to use here is an HTTPCaller transformer to first download the file and then read the downloaded file with a FeautreReader.
Example Attached.
The HTTPCaller method will probably work for link 2 as well
Good Luck
This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
For Link one it looks like the data is being compiled based on a request. As a tip you can open up the develper tap in your browser and look at the network traffic when you hit the butten. You will likely see some kind of query. (URL + some settings) The settings are set based on user the user input like published parameters in an FME workspace so the the download url is not static
Here's a screen shot of what I mean from looking at link 1
Here at the top or the right window you can see the base URL. At the bottom are the request parameters. If you switch the view from "From Data" to "View Source" you can see the end part of the URL
There is another method. Just because we have the URL here unfortunately when putting it in a Reader FME removes all the stuff off the end and adds a .gpkg extension to the URL.
What you want to use here is an HTTPCaller transformer to first download the file and then read the downloaded file with a FeautreReader.
Example Attached.
The HTTPCaller method will probably work for link 2 as well
Both sites you've listed appear to have been develop by off road software. I would try to contact off road software and ask if they offer an API to their websites and data sources (usually a REST API).
Here at the top or the right window you can see the base URL. At the bottom are the request parameters. If you switch the view from "From Data" to "View Source" you can see the end part of the URL
