Skip to main content
Question

Write a web page source code to a file in order to extract lat and lon, to then write to a TAB file.

  • October 31, 2016
  • 4 replies
  • 74 views

Forum|alt.badge.img+3

Hi,

I want to produce some points on a map (writing into a TAB file) based on the lat and lon location which is buried in the source code of a web page.

e.g. www.cqc.org.uk/location/1-2634447449/contact

Does anyone know if this doable using Desktop?

Cheers,

Jake

This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

4 replies

itay
Supporter
Forum|alt.badge.img+19
  • Supporter
  • October 31, 2016

Hi, Dont know if you can get to the coordinates, alternatively you can try to parse the HTML and get the address for geocoding.


redgeographics
VIP
Forum|alt.badge.img+62

Yes, that's possible. I checked the source of that page and noticed a set of lat/lon coordinates in one of the header lines:

<script>jQuery.extend(Drupal.settings, {"basePath":"\/","pathPrefix":"","cqc_data_contact":{"lat":"53.7226304","lon":"-1.8862726","location_address":"Gibbet Street,\u003Cbr \

And that appears to be JSON, which is a format that FME can handle!

So I set up a workspace to read the .html file as a text file, then filter out that line using a StringSearcher, clean it up a bit in order to isolate just the JSON content, run it through a JSONFragmenter and have it expose the attributes lat and lon, then create a point out of that.

knipsel 

locationscraper.fmwt 

You probably need some more work to output useful attributes, but the basics (scraping the lat/lon from the html code and creating a point) are there. Hope this helps.


itay
Supporter
Forum|alt.badge.img+19
  • Supporter
  • October 31, 2016

Yes, that's possible. I checked the source of that page and noticed a set of lat/lon coordinates in one of the header lines:

<script>jQuery.extend(Drupal.settings, {"basePath":"\/","pathPrefix":"","cqc_data_contact":{"lat":"53.7226304","lon":"-1.8862726","location_address":"Gibbet Street,\u003Cbr \

And that appears to be JSON, which is a format that FME can handle!

So I set up a workspace to read the .html file as a text file, then filter out that line using a StringSearcher, clean it up a bit in order to isolate just the JSON content, run it through a JSONFragmenter and have it expose the attributes lat and lon, then create a point out of that.

knipsel 

locationscraper.fmwt 

You probably need some more work to output useful attributes, but the basics (scraping the lat/lon from the html code and creating a point) are there. Hope this helps.

 

nice one @redgeographics, completly missed out the coords.

 


Forum|alt.badge.img+3
  • Author
  • October 31, 2016

This is great, thanks a lot!