Hi there I have a viewer https://maps.greydc.govt.nz/intramaps90/default.htm?configId=2f6f83a4-782b-45eb-9b60-d5819e718428 where I wish to extract the water assets including water, stormwater and wastewater. I would like some guidance in how I can scrape/extract this data. Can it be done?
Solved
Extracting Data from Intramap viewer
+9- Contributor
- 28 replies
Best answer by jlbaker2779
It's a WMS, so you really can't do much other than download the tiles since they have denied access to the mapserver itself. So, this is a fun one!
Using Fiddler I got one of the requests for the map and then built two workbenches to download the images and mosaic them together.
Example web request Link
Using that URL you can get the XY coordinates of the bottom left corner of the image via the "&BBOX=1469000,5307000,1470000,5308000" parameter, use a 2DGridCreator with the XY for the bottom left set and build 1000x1000 ground unit image request URLs to sent to the server, sent the request to the server and save the files with @value(_row)-@value(_column) as the name for merging in the second workbench.
After the images are downloaded you can then bring them in to FME, merge the data by the row/column attribute from the filename and grid to get the coordinates, the send them through the Offsetter. It will create a mosaic.
Results
Mosaic of 4X4 (16 tile) sample download
In most cases you'll need to georeference the imagine into Arcmap. When that is done, you can use FME once again to create lines from the raster based off pixel values.
Attached are the two workbenches and sample data. All you have to do is identify a coordinate on the map at the bottom left of the screen, top right, math it, and then put in how many tiles you want created in the 2DGridCreator. Make sure to change both grid creators in both workbenches.
This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
+2It's a WMS, so you really can't do much other than download the tiles since they have denied access to the mapserver itself. So, this is a fun one!
Using Fiddler I got one of the requests for the map and then built two workbenches to download the images and mosaic them together.
Example web request Link
Using that URL you can get the XY coordinates of the bottom left corner of the image via the "&BBOX=1469000,5307000,1470000,5308000" parameter, use a 2DGridCreator with the XY for the bottom left set and build 1000x1000 ground unit image request URLs to sent to the server, sent the request to the server and save the files with @value(_row)-@value(_column) as the name for merging in the second workbench.
After the images are downloaded you can then bring them in to FME, merge the data by the row/column attribute from the filename and grid to get the coordinates, the send them through the Offsetter. It will create a mosaic.
Results
Mosaic of 4X4 (16 tile) sample download
In most cases you'll need to georeference the imagine into Arcmap. When that is done, you can use FME once again to create lines from the raster based off pixel values.
Attached are the two workbenches and sample data. All you have to do is identify a coordinate on the map at the bottom left of the screen, top right, math it, and then put in how many tiles you want created in the 2DGridCreator. Make sure to change both grid creators in both workbenches.