Solved
HTTPCaller, zillow
Good morning! I am having troubles with the HTTPCaller/HTMLExtractor. I am trying to pull the zestimate values from Zillow.com for a specific house (right now it's just some random house I picked). The HTTPCaller doesn't seem to be pulling "real" information in it's _response_body attribute field. I've been following the example here: http://docs.safe.com/fme/2018.0/html/FME_Desktop_Documentation/FME_Transformers/Transformers/htmlextractor.htm @trentatsafe suggested: "It appears that the HTTP response is looking for verification that it is not a computer. Do you have an account? If so, what is the reponse you get when using your login credentials as the authentication for the HTTPCALLER "I tried adding my login - but got the same results. It's also a bit odd that it is looking for a login, as you don't typically have to login to Zillow to see their values. Any suggestions?
Best answer by jillmperi
Thanks @NathanAtSafe,
I did find a work around...some notes for anyone else working on this:
There is a Zillow API Network that you have to sign up for to get a key code [your ID]. I was then able to "create" the websites for my specific properties from: http://www.zillow.com/webservice/GetZestimate.htm?zws-id=[your ID]&ZPID;=[your property Zpid]
(example from here: https://www.youtube.com/watch?v=JhiT0Ty5YSc )
I was then able to use the HTTPCaller using that "created" website and the XMLFlattener to extract the zestimate (be sure to set Attributes to Expose to amount) and work from there.
Thanks @NathanAtSafe and @trentatsafe for your help! Hope these notes help someone else.
This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
+9- Safer
- 284 replies
Hi @jillmperi,
From the JavaScript spit into the response body, it looks like the Zillow page has some scripts protecting against scraping their website for data. Trying a few other URLs gives the same result - a captcha test is required. You might try rooting around the web page with the Developer console in Chrome open to try to understand the page source code, or perhaps get in touch with Zillow about how to work with their APIs.
Hope this helps!
Nathan
Thanks @NathanAtSafe,
I did find a work around...some notes for anyone else working on this:
There is a Zillow API Network that you have to sign up for to get a key code [your ID]. I was then able to "create" the websites for my specific properties from: http://www.zillow.com/webservice/GetZestimate.htm?zws-id=[your ID]&ZPID;=[your property Zpid]
(example from here: https://www.youtube.com/watch?v=JhiT0Ty5YSc )
I was then able to use the HTTPCaller using that "created" website and the XMLFlattener to extract the zestimate (be sure to set Attributes to Expose to amount) and work from there.
Thanks @NathanAtSafe and @trentatsafe for your help! Hope these notes help someone else.
+9- Safer
- 284 replies
Thanks @NathanAtSafe,
I did find a work around...some notes for anyone else working on this:
There is a Zillow API Network that you have to sign up for to get a key code [your ID]. I was then able to "create" the websites for my specific properties from: http://www.zillow.com/webservice/GetZestimate.htm?zws-id=[your ID]&ZPID;=[your property Zpid]
(example from here: https://www.youtube.com/watch?v=JhiT0Ty5YSc )
I was then able to use the HTTPCaller using that "created" website and the XMLFlattener to extract the zestimate (be sure to set Attributes to Expose to amount) and work from there.
Thanks @NathanAtSafe and @trentatsafe for your help! Hope these notes help someone else.
Nathan