Skip to main content
Solved

Unable to Extract Links from Webpage using HTMLExtractor

  • September 30, 2024
  • 2 replies
  • 46 views

joshuajames1979
Contributor
Forum|alt.badge.img+7

Hi there, I am trying to extract links from this page Mornington Peninsula Planning Scheme - Maps using the methods posted on this forum as well as from the official documentation HTMLExtractor (safe.com), however there is zero total features read, meaning no output. Would this be due to the way the site is structured? I am specifally trying to extract the urls for the PDF maps on the left, and output to a SQL table and link to our GIS, as these links change every so often.

Regards,

Josh 

Best answer by todd_davis

Hi Josh,

You are correct that the htmlextractor is not the best used in this webpage. If you have a look at the calls that are happening (Developer Tools in your browser), you will see a call that has all this detail:

https://api.app.planning.vic.gov.au/planning/v2/pdfmaps/scheme/morn

Which returns json and you can break that down.

The attached process will give you all the details in that pane, and you can use the “url” attribute added to https://planning-schemes.app.planning.vic.gov.au to get all the pdfs

 

Cheers,

Todd

View original
Did this help you find an answer to your question?

2 replies

todd_davis
Influencer
Forum|alt.badge.img+22
  • Influencer
  • Best Answer
  • September 30, 2024

Hi Josh,

You are correct that the htmlextractor is not the best used in this webpage. If you have a look at the calls that are happening (Developer Tools in your browser), you will see a call that has all this detail:

https://api.app.planning.vic.gov.au/planning/v2/pdfmaps/scheme/morn

Which returns json and you can break that down.

The attached process will give you all the details in that pane, and you can use the “url” attribute added to https://planning-schemes.app.planning.vic.gov.au to get all the pdfs

 

Cheers,

Todd


joshuajames1979
Contributor
Forum|alt.badge.img+7

Thanks Todd. That’s great. I’ll make a couple of tweaks and I can use this to output to a SQL table.

Thanks again.

Josh


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings