Hi there I have a viewer https://maps.greydc.govt.nz/intramaps90/default.htm?configId=2f6f83a4-782b-45eb-9b60-d5819e718428 where I wish to extract the water assets including water, stormwater and wastewater. I would like some guidance in how I can scrape/extract this data. Can it be done?
Solved
Extracting Data from Intramap viewer
+9- Contributor
Best answer by jlbaker2779
It's a WMS, so you really can't do much other than download the tiles since they have denied access to the mapserver itself. So, this is a fun one!
Using Fiddler I got one of the requests for the map and then built two workbenches to download the images and mosaic them together.
Example web request Link
Using that URL you can get the XY coordinates of the bottom left corner of the image via the "&BBOX=1469000,5307000,1470000,5308000" parameter, use a 2DGridCreator with the XY for the bottom left set and build 1000x1000 ground unit image request URLs to sent to the server, sent the request to the server and save the files with @value(_row)-@value(_column) as the name for merging in the second workbench.
After the images are downloaded you can then bring them in to FME, merge the data by the row/column attribute from the filename and grid to get the coordinates, the send them through the Offsetter. It will create a mosaic.
Results
Mosaic of 4X4 (16 tile) sample download
In most cases you'll need to georeference the imagine into Arcmap. When that is done, you can use FME once again to create lines from the raster based off pixel values.
Attached are the two workbenches and sample data. All you have to do is identify a coordinate on the map at the bottom left of the screen, top right, math it, and then put in how many tiles you want created in the 2DGridCreator. Make sure to change both grid creators in both workbenches.
+2It's a WMS, so you really can't do much other than download the tiles since they have denied access to the mapserver itself. So, this is a fun one!
Using Fiddler I got one of the requests for the map and then built two workbenches to download the images and mosaic them together.
Example web request Link
Using that URL you can get the XY coordinates of the bottom left corner of the image via the "&BBOX=1469000,5307000,1470000,5308000" parameter, use a 2DGridCreator with the XY for the bottom left set and build 1000x1000 ground unit image request URLs to sent to the server, sent the request to the server and save the files with @value(_row)-@value(_column) as the name for merging in the second workbench.
After the images are downloaded you can then bring them in to FME, merge the data by the row/column attribute from the filename and grid to get the coordinates, the send them through the Offsetter. It will create a mosaic.
Results
Mosaic of 4X4 (16 tile) sample download
In most cases you'll need to georeference the imagine into Arcmap. When that is done, you can use FME once again to create lines from the raster based off pixel values.
Attached are the two workbenches and sample data. All you have to do is identify a coordinate on the map at the bottom left of the screen, top right, math it, and then put in how many tiles you want created in the 2DGridCreator. Make sure to change both grid creators in both workbenches.
Reply
Rich Text Editor, editor1
Editor toolbars
Press ALT 0 for help
Helpful Members This Week
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.