Hello everyone,
Today I have a Question of the Week that came through Live Chat.
Question of the Week
Here's a section of the Live Chat log:
Andrea E: Hi, how are you today?
User: Hi Andrea
Andrea E: How can I help you?
User: I am doing a spatial overlay of a point layer (animal licenses) over a postcode polygon
User: I was able to get the total number of point inside each postcode
User: but I also want to get the total number per animal type - dog or cat
Andrea E: So you want the break down within each postcode of dog or cat?
So the question is: how many of each type of animal are licensed for each postcode boundary? For that we'll need to do an overlay, maybe create a list, and then calculate some statistics.
Live Chat
So Live Chat is how our support teams can communicate with users in an online conversation. I don't do that work and I admire those who do because you have to think very quickly to come up with an answer. In the time she had, Andrea came up with this solution:
Using Businesses/Neighborhoods (instead of Animals/Postcodes) she carries out an overlay first of all. That returns a total count per neighborhood, but not a count of each business type; and you can't group-by type because the area features don't have a matching attribute.
So... the businesses - now with added postcode attribute - are then run through a StatisticsCalculator and Aggregator to aggregate business types per postcode. That's simple enough, but then we need more transformers to restructure the attributes and attach them back to the area features.
It's a nice solution, but it's a bit slow around the Aggregator transformer (5+ minutes). So is there a better way?
Using Lists
If we check the PointOnAreaOverlayer:Area output, we see that each neighborhood gets a list containing businesses and types:
Could we calculate the numbers from this? The ListElementCounter would count all elements (it has no group-by option) and the ListSummer has no numeric values to work on. But there is one transformer that will do what we want: the ListHistogrammer!
The ListHistogrammer
This excellent little transformer is exactly like a group-based ListElementCounter. It will count how many of each business (or animal) type there are in the list of overlapping point features:
So now we have the answer. There are 89 restaurants, 62 offices, and 39 residential/commercial businesses in this neighborhood. Hurrah!
The problem is, the results are still in a list, and we need attributes. How do we get that? A ListExploder would give us multiple features and an AttributeCopier would need to be hard-coded (we couldn't know in advance how many entries were in the list).
So what to do?
The FME Hub Comes to the Rescue!
When you can't find an FME transformer, look on the FME Hub! There I found a ListBreaker transformer, which almost did what I wanted, but not quite (I even created that, but couldn't remember it); but the one that solved the problem was the ListKeyValuePairExtractor.
This excellent transformer - created and submitted by community member
@geosander - takes a list of names/values and splits it up into attributes. With it I get this:
Just what we need; in only 2 transformers and 3 seconds processing time.
Summary
What are the points to take from this post? After all, I can't preach a sermon without having a meaning! There are three things I want you to remember:
- The ListHistogrammer is a great transformer for calculating statistics, and perhaps not very well known (ranks 209 out of 500 transformers). If you didn't know about it, go check it out.
- If you can't find an FME transformer to do what you need, look at the FME Hub. You don't even need to go to the web site. Just do a Quick Add search (I typed "list") and see what comes up under the Hub section. Often someone else has had the same problem and shared their solution.
- This really underlines the need to do "Code Reviews" on FME workspaces. The user had a method but said "I feel like this could be done more elegantly." Andrea looked and created an improved version. I looked and improved on that, and I wouldn't be surprised if one of you looked and improved my workspace. The point is, get a colleague to review your workspaces before going into production, and if you don't have a colleague, post them to the community. There's always someone willing to help.
I hope you found this little post interesting.
Thank you to both of you!
I've decided to choose an easier solution (well, for me, i think).
I'll use the SystemCaller in my workbench, where I can put my UNC/user ID. As described here
It works great.
I don't want to change my account permissions, it wouldn't be useful anyway cuz I have lots of different users on different servers.
