Skip to main content
  • EN

Hi FME'ers,

In an hour or so we're starting the webinar on Choosing the Right Transformer (24-May, 10am PDT). Several users already submitted scenarios where they are facing a challenge finding the right transformer, and Dale, Iris, and I will try and live-build a workspace to solve their issue.

At the same time, we had a few scenarios that we won't have time to cover in full, and wanted to throw them out for your suggestions. So, if you think you know what transformers to use in the following scenarios, or if you have a brief scenario that we can try and demonstrate online, please let us know in the comments below.

We'll do our best to work this content into the webinar to make it really live (and quite exciting for us presenters!) - and if you have any comments on the webinar or suggestions for the challenges we are showing, do also let us know.

Cheers

Mark

Scenario 1: Given this feed of live cycle data from TFL (Transport for London), which transformers would you suggest to read the data, to create point features, to remove extra attributes, and to add/update XML metadata using a specific schema? Is there a specific technique that you can recommend?

Scenario 2: Given a webpage (no demo available unfortunately) with a list of video files, which transformers would you suggest to download each file, compare it to a list of existing files, and (if new) publish it to a YouTube channel? What other files could be published using a Google API?

I submitted #2 so I thought I add my solution here for reference.

First off I build a web scraper using http caller and html extractor.

I ended up not using the this method and switching to an API to get this data.

I can post more of this solution if anyone wants but I'm still finishing off the the youtube upload due to an issue outside of FME.

Scenario 1: (I know its a bit late)

To read the data, to create point features and choose attributes I would use a Python Creator with following code:

import fmeobjects
import requests
from xml.etree import ElementTree as ET

# Template Class
class FeatureCreator(object):
    def __init__(self):
        pass
    def input(self,feature):
        Path = r"https://tfl.gov.uk/tfl/syndication/feeds/cycle-hire/livecyclehireupdates.xml"
	colList = /"name", "terminalName","lat","long","nbBikes","nbEmptyDocks","nbDocks"]
        response = requests.get(Path)
        root = ET.fromstring(response.content)

        for child in root:
            newFeature = fmeobjects.FMEFeature()
            for grandchild in child:
		if grandchild.tag in colList: 
                	newFeature.setAttribute(grandchild.tag, grandchild.text)
            self.pyoutput(newFeature)
    def close(self):
        pass 

I thought I'd just upload the first bit of XML reading that I created during the webinar. It doesn't address the metadata, but it reads the feed and splits the data into attributes.

I used a Textfile reader with the parameter set to read the entire feed into a single attribute, and then fragmented the data with an XMLFragmenter transformer.

The XMLFragmenter will read directly from the XML feed, but I used a text reader because then I could do a partial run on the XMLFragmenter without having to reread the feed.

Workspace - uses the latest 2018.0 version (but should be OK with any 2018.0)

Reply


Community Stats

30,162
Posts
114,480
Replies
37,807
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.