Skip to main content
Question

Data Scraping with Download

  • April 7, 2017
  • 3 replies
  • 66 views

Forum|alt.badge.img

Does anyone have a routine or know how to build a routine to access a webpage, make a selection from a dropdown menu, and then select save at a pop up window? I'm sure this can be done in Python, but I do not have the background.

This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

3 replies

mygis
Supporter
Forum|alt.badge.img+14
  • Supporter
  • 307 replies
  • June 27, 2017

Hi @rick_williams,

May I know what kind of information you would like to retrieve from the web page? Are those tables?


mark2atsafe
Safer
Forum|alt.badge.img+57
  • Safer
  • 2554 replies
  • June 27, 2017
If it's data served by FME Server then the server API would be a good solution. Otherwise try examine the code for the page and see if the menu can be incorporated somehow into a call from a HTTPCaller transformer. Though how you save content from a pop-up window I don't know.

 


stalknecht
Contributor
Forum|alt.badge.img+21
  • Contributor
  • 305 replies
  • July 14, 2017

In google chrome you can use developer tools to inspect the calls that are going. The HTTPCaller is your friend to use these. There is also the "HTML Table" - Reader that can read html tables from a webpage. If you really want to do mouse-clicks then you can try to download a mouse-recorder and call it with the SystemCaller transformer. A FeatureReader can be used to get the data back into the workspace.