Getting parameters from a URL

Hi Jonathan,

 

 

The quick and easy solution would be to use a couple of AttibuteSplitters and see how it works for you.

 

 

An intermediary solution would be to use a StringSearcher where you use regexp grouping "(...)" to isolate specific values into their corresponding attributes via the "matched parts" list.

 

 

The advanced solution would be to use a PythonCaller with something like the urlparse module.

 

 

For the two first solutions, you might also want to consider using a TextDecoder to "unpack" any potentially URLEncoded text first.

 

 

David

Hi Jonathan,

 

 

depending on the requirement and the condition of the URL string, different approaches can be considered. When names of all attributes which should be extracted are known, I would use the StringSearcher transformer.

 

If the order of attributes in a URL string is invariable, using single StringSearcher could be the quickest solution: ^.+\\?TRANSPARENT=(.*)&SERVICE=(.*)&VERSION=(.*)<others>$   If not, use multiple StringSearchers for every attribute one by one: ^.+[\\?|&]TRANSPARENT=([^&]*).*$ ^.+[\\?|&]SERVICE=([^&]*).*$ ^.+[\\?|&]VERSION=([^&]*).*$ etc.   Otherwise, following David's suggestion, I would use a PythonCaller.

 

Takashi

Thanks for the replies, I'll give them a shot. I am surprised there isn't a transformer for this, especially given how FME has been ever more web-ified over the past few versions.

 

Cheers,

 

Jonathan

For posterity, here is a script for a PythonCaller that will split the attribute "URL" (upper case) into its individual components.

 

 

-----

 

import fmeobjects from urlparse import urlparse, parse_qs   def URLParser(feature):     url = feature.getAttribute("URL")     if url:

 

        # Parse url and query string         parsed = urlparse(url)         params = parse_qs(parsed.query)

 

        # Set host and path         feature.setAttribute("URL_HOST", parsed.netloc)         feature.setAttribute("URL_PATH", parsed.path)

 

 

        # Create an attribute for all query parameters, prefixed by "URL_"         for param in params.keys():             feature.setAttribute("URL_"+param, params[param][0])

 

-----

 

 

Given the URL: "http://www.test.org/geoserver/ows?TRANSPARENT=TRUE&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetLegendGraphic&EXCEPTIONS=application%2Fvnd.ogc.se_xml&LAYER=ROAD_SECS_BY_CLASS_WSHIRE&FORMAT=image%2Fpng&SCALE=19999.99999369495"

 

 

it will output the following new attributes:

 

 

URL_EXCEPTIONS -> 'application/vnd.ogc.se_xml' URL_FORMAT -> 'image/png' URL_HOST -> 'www.test.org' URL_LAYER -> 'ROAD_SECS_BY_CLASS_WSHIRE' URL_PATH -> '/geoserver/ows' URL_REQUEST -> 'GetLegendGraphic' URL_SCALE -> '19999.99999369495' URL_SERVICE -> 'WMS' URL_TRANSPARENT -> 'TRUE' URL_VERSION -> '1.1.1'

 

 

David

Thanks David, I'll be sure to give it a try!

Hi David,

 

Your code works great. One improvement though -

 

 

            feature.setAttribute("URL_"+param.upper(), params[param][0])

 

 

This way it doesn't matter what case the parameter is in the URL, it will always be upper case in the FME Attribute. Important as FME is case sensitive.

 

 

Thanks again!

 

Jonathan