Skip to main content

Hi,

I want to produce some points on a map (writing into a TAB file) based on the lat and lon location which is buried in the source code of a web page.

e.g. www.cqc.org.uk/location/1-2634447449/contact

Does anyone know if this doable using Desktop?

Cheers,

Jake

Hi, Dont know if you can get to the coordinates, alternatively you can try to parse the HTML and get the address for geocoding.


Yes, that's possible. I checked the source of that page and noticed a set of lat/lon coordinates in one of the header lines:

<script>jQuery.extend(Drupal.settings, {"basePath":"\/","pathPrefix":"","cqc_data_contact":{"lat":"53.7226304","lon":"-1.8862726","location_address":"Gibbet Street,\u003Cbr \

And that appears to be JSON, which is a format that FME can handle!

So I set up a workspace to read the .html file as a text file, then filter out that line using a StringSearcher, clean it up a bit in order to isolate just the JSON content, run it through a JSONFragmenter and have it expose the attributes lat and lon, then create a point out of that.

knipsel 

locationscraper.fmwt 

You probably need some more work to output useful attributes, but the basics (scraping the lat/lon from the html code and creating a point) are there. Hope this helps.


Yes, that's possible. I checked the source of that page and noticed a set of lat/lon coordinates in one of the header lines:

<script>jQuery.extend(Drupal.settings, {"basePath":"\/","pathPrefix":"","cqc_data_contact":{"lat":"53.7226304","lon":"-1.8862726","location_address":"Gibbet Street,\u003Cbr \

And that appears to be JSON, which is a format that FME can handle!

So I set up a workspace to read the .html file as a text file, then filter out that line using a StringSearcher, clean it up a bit in order to isolate just the JSON content, run it through a JSONFragmenter and have it expose the attributes lat and lon, then create a point out of that.

knipsel 

locationscraper.fmwt 

You probably need some more work to output useful attributes, but the basics (scraping the lat/lon from the html code and creating a point) are there. Hope this helps.

 

nice one @redgeographics, completly missed out the coords.

 


This is great, thanks a lot!


Reply