Hi there I would like to scrape the data from this page. So want the lat/longs of the pins on the map and the background water outage data. I can't find an easy way to scrape the data and am just going around in circles trying to extract the data.
Hi there I would like to scrape the data from this page. So want the lat/longs of the pins on the map and the background water outage data. I can't find an easy way to scrape the data and am just going around in circles trying to extract the data.
It looks like the site you are trying to access is Javascript based. Unfortunately, there is not much FME can do to retrieve data directly from that URL. If the site has an API, you will be likely able to access data through their API endpoints, using HTTPCaller.
Scraping data from JavaScript-based sites, like the one you mentioned, can be a bit tricky since the content is often loaded dynamically. Here are a few steps you can take to extract the lat/longs of the pins and the related data:
To handle the complexity of web scraping from JavaScript-based sites while managing multiple web profiles, consider using tools like an anti-detect browser such as Multilogin. It allows you to manage multiple browsing environments, ensuring anonymity and efficient data scraping while navigating complex JavaScript-based sites.
If you have any specific data-related questions or need help with tools, feel free to ask!
This appears to be doable. All map data is available as GEOJSON inside the HTML page.
As all the GEOJSON is contained within one line, it is easier to read the HTML page as a Text file, and identify the right feature.
Some string juggling followed by some JSON processing gets you the features of the map.
See the attached workspace. Many attributes still need to be exposed.