Skip to main content
Question

Data Scraping with Download


Forum|alt.badge.img

Does anyone have a routine or know how to build a routine to access a webpage, make a selection from a dropdown menu, and then select save at a pop up window? I'm sure this can be done in Python, but I do not have the background.

3 replies

mygis
Supporter
Forum|alt.badge.img+13
  • Supporter
  • June 27, 2017

Hi @rick_williams,

May I know what kind of information you would like to retrieve from the web page? Are those tables?


mark2atsafe
Safer
Forum|alt.badge.img+44
If it's data served by FME Server then the server API would be a good solution. Otherwise try examine the code for the page and see if the menu can be incorporated somehow into a call from a HTTPCaller transformer. Though how you save content from a pop-up window I don't know.

 


stalknecht
Contributor
Forum|alt.badge.img+19
  • Contributor
  • July 14, 2017

In google chrome you can use developer tools to inspect the calls that are going. The HTTPCaller is your friend to use these. There is also the "HTML Table" - Reader that can read html tables from a webpage. If you really want to do mouse-clicks then you can try to download a mouse-recorder and call it with the SystemCaller transformer. A FeatureReader can be used to get the data back into the workspace.


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings