Question

Write a web page source code to a file in order to extract lat and lon, to then write to a TAB file.

  • 31 October 2016
  • 4 replies
  • 1 view

Badge +3

Hi,

I want to produce some points on a map (writing into a TAB file) based on the lat and lon location which is buried in the source code of a web page.

e.g. www.cqc.org.uk/location/1-2634447449/contact

Does anyone know if this doable using Desktop?

Cheers,

Jake


4 replies

Badge +16

Hi, Dont know if you can get to the coordinates, alternatively you can try to parse the HTML and get the address for geocoding.

Userlevel 5
Badge +25

Yes, that's possible. I checked the source of that page and noticed a set of lat/lon coordinates in one of the header lines:

<script>jQuery.extend(Drupal.settings, {"basePath":"\/","pathPrefix":"","cqc_data_contact":{"lat":"53.7226304","lon":"-1.8862726","location_address":"Gibbet Street,\u003Cbr \

And that appears to be JSON, which is a format that FME can handle!

So I set up a workspace to read the .html file as a text file, then filter out that line using a StringSearcher, clean it up a bit in order to isolate just the JSON content, run it through a JSONFragmenter and have it expose the attributes lat and lon, then create a point out of that.

knipsel 

locationscraper.fmwt 

You probably need some more work to output useful attributes, but the basics (scraping the lat/lon from the html code and creating a point) are there. Hope this helps.

Badge +16

Yes, that's possible. I checked the source of that page and noticed a set of lat/lon coordinates in one of the header lines:

<script>jQuery.extend(Drupal.settings, {"basePath":"\/","pathPrefix":"","cqc_data_contact":{"lat":"53.7226304","lon":"-1.8862726","location_address":"Gibbet Street,\u003Cbr \

And that appears to be JSON, which is a format that FME can handle!

So I set up a workspace to read the .html file as a text file, then filter out that line using a StringSearcher, clean it up a bit in order to isolate just the JSON content, run it through a JSONFragmenter and have it expose the attributes lat and lon, then create a point out of that.

knipsel 

locationscraper.fmwt 

You probably need some more work to output useful attributes, but the basics (scraping the lat/lon from the html code and creating a point) are there. Hope this helps.

 

nice one @redgeographics, completly missed out the coords.

 

Badge +3

This is great, thanks a lot!

Reply