Question

Can FME find the host location of a website?

+6

I am wanting to feed in a number of web urls and I would like to find the host / server location (and IP address ideally) for the sites.

I seem to remember seeing this done at a conference on FME.

Thanks

Maybe you could use something like this service: http://ip-api.com/docs/

To split up an URL into the different bits and pieces you could use the "urlparse" Python module. It is mentioned in this old post, but unfortunately the code samples have been a bit mangled by an upgrade in the forum software: https://knowledge.safe.com/questions/2581/getting-parameters-from-a-url.html

+21

Hi Seb,

Perhaps you could try our custom transformer the IPGeocoder to do this.

+6

Many thanks both 1spatialdave and david_r.

I have gone with the ip-api.com version as it gives more detail on the ISP.

Many thanks

Seb

david_r wrote:

Maybe you could use something like this service: http://ip-api.com/docs/

To split up an URL into the different bits and pieces you could use the "urlparse" Python module. It is mentioned in this old post, but unfortunately the code samples have been a bit mangled by an upgrade in the forum software: https://knowledge.safe.com/questions/2581/getting-parameters-from-a-url.html

I just saw this on their website:

Our system will automatically ban any IP addresses doing over 150 requests per minute.

If that might be an issue, you could use a Decelerator to avoid hitting the limit, e.g. this will limit yourself to maximum 120 lookups per minute:

+6

david_r wrote:

Maybe you could use something like this service: http://ip-api.com/docs/

To split up an URL into the different bits and pieces you could use the "urlparse" Python module. It is mentioned in this old post, but unfortunately the code samples have been a bit mangled by an upgrade in the forum software: https://knowledge.safe.com/questions/2581/getting-parameters-from-a-url.html

Thanks david_r

I spotted that as well. I don't think I will be hitting it with 150 requests a minute.

Thanks

Seb