Question

Can FME find the host location of a website?

  • 7 September 2016
  • 5 replies
  • 1 view

Badge +1

I am wanting to feed in a number of web urls and I would like to find the host / server location (and IP address ideally) for the sites.

I seem to remember seeing this done at a conference on FME.

Thanks


5 replies

Userlevel 4

Maybe you could use something like this service: http://ip-api.com/docs/

To split up an URL into the different bits and pieces you could use the "urlparse" Python module. It is mentioned in this old post, but unfortunately the code samples have been a bit mangled by an upgrade in the forum software: https://knowledge.safe.com/questions/2581/getting-parameters-from-a-url.html

Badge +10

Hi Seb,

Perhaps you could try our custom transformer the IPGeocoder to do this.

https://hub.safe.com/transformers/ipgeocoder

Badge +1

Many thanks both 1spatialdave and david_r.

I have gone with the ip-api.com version as it gives more detail on the ISP.

Many thanks

Seb

Userlevel 4

Maybe you could use something like this service: http://ip-api.com/docs/

To split up an URL into the different bits and pieces you could use the "urlparse" Python module. It is mentioned in this old post, but unfortunately the code samples have been a bit mangled by an upgrade in the forum software: https://knowledge.safe.com/questions/2581/getting-parameters-from-a-url.html

I just saw this on their website:

 

Our system will automatically ban any IP addresses doing over 150 requests per minute.

 

 

If that might be an issue, you could use a Decelerator to avoid hitting the limit, e.g. this will limit yourself to maximum 120 lookups per minute:

 

 

 

 

Badge +1

Maybe you could use something like this service: http://ip-api.com/docs/

To split up an URL into the different bits and pieces you could use the "urlparse" Python module. It is mentioned in this old post, but unfortunately the code samples have been a bit mangled by an upgrade in the forum software: https://knowledge.safe.com/questions/2581/getting-parameters-from-a-url.html

 

Thanks david_r

 

I spotted that as well. I don't think I will be hitting it with 150 requests a minute.

 

 

Thanks

 

Seb

Reply