Question

PointOnAreaOverlay and TestFilter

sunsilk11
Rank
Badge +6

Hi I have a point and polygon layer. I need to populate different counts of points in each polygon satisfying different attribute queries on the point layer.

e.g count of supermarkets, sports centre in each polygons. I have 8 such queries, so would populate 8 fields in the polygon layer.

I tried using TestFilter on point fc then PointOnAreaOverlay, but this doesn't work as I cannot get multiple If and If statements from TestFilter. Instead It gives me If Else If.

I would like to avoid having 8 different TestFilters and PointOnAreaOverlay transformers if possible.

Any ideas?

34 replies

austinh
Rank
Badge +4

The PointOnArea overlay creates a field called _overlaps. This should have the count of points on your polygon. You can use the TestFilter after the PiintOnOverlay transformer to test If the _overlap field is >0. You could also use an AttributeFilter and creat an output for 8 different possibilities

takashi
Rank
Userlevel 2
Badge +17

Hi @sunsilk11, this workflow might help you. Assuming that the point features have an attribute storing type of the point (e.g. "_point_type" that stores one of "supermarket", "sports centre" etc.) and the polygon features have a unique ID attribute (e.g. "_polygon_id").

  1. Check the Merge Attributes option in the PointOnAreaOverlayer, in order to merge attributes of a polygon to ovelaid points.
  2. Add an attribute whose name is equal to the value of "_point_type" and whose value is 1 to the point features.
  3. Expose those new attribute names (i.e. "supermarket", "sports centre", etc.).
  4. Aggregate the point features grouping by "_polygon_id", to sum the values of those attributes for each polygon.

[Addition] Then, you can merge the result to polygons using "_polygon_id" as the join key if necessary.

takashi
Rank
Userlevel 2
Badge +17

Hi @sunsilk11, this workflow might help you. Assuming that the point features have an attribute storing type of the point (e.g. "_point_type" that stores one of "supermarket", "sports centre" etc.) and the polygon features have a unique ID attribute (e.g. "_polygon_id").

  1. Check the Merge Attributes option in the PointOnAreaOverlayer, in order to merge attributes of a polygon to ovelaid points.
  2. Add an attribute whose name is equal to the value of "_point_type" and whose value is 1 to the point features.
  3. Expose those new attribute names (i.e. "supermarket", "sports centre", etc.).
  4. Aggregate the point features grouping by "_polygon_id", to sum the values of those attributes for each polygon.

[Addition] Then, you can merge the result to polygons using "_polygon_id" as the join key if necessary.

0684Q00000ArL37QAF.png

Alternatively, List/JSON operations could also be applied effectively.

 

0684Q00000ArM52QAF.png

JSONTemplater | Root Template

 

{|
    let $count := fme:get-list-attribute("_histogram{}.count")
    for $type at $i in fme:get-list-attribute("_histogram{}.value")
    return {$type : $count[$i]}
|}


 

sunsilk11
Rank
Badge +6

@takashI

I don't have an attribute which denote supermarket etc but instead need to apply query like classification code in('a','b',...) to get all supermarket point features.

I'll look into using the JSON/List options

Thank you

takashi
Rank
Userlevel 2
Badge +17

@takashI

I don't have an attribute which denote supermarket etc but instead need to apply query like classification code in('a','b',...) to get all supermarket point features.

I'll look into using the JSON/List options

Thank you

How can you determine what kind of place (supermarket, sport center, etc.) a point represents?

 

 

sunsilk11
Rank
Badge +6

@takashI

I don't have an attribute which denote supermarket etc but instead need to apply query like classification code in('a','b',...) to get all supermarket point features.

I'll look into using the JSON/List options

Thank you

I would get the kind of point features using attribute queries e.g. classification_code in ('a', 'b',..) for supermarkets

 

 

takashi
Rank
Userlevel 2
Badge +17

@takashI

I don't have an attribute which denote supermarket etc but instead need to apply query like classification code in('a','b',...) to get all supermarket point features.

I'll look into using the JSON/List options

Thank you

If the mapping rule between code and class name ("_point_type" in the workflow I've posted) is static, the AttributeValueMapper would be a simple way to create a new attribute storing class name corresponding to a code.

 

Alternatively, you can also create an external table (CSV, Excel, or so) that defines the mapping rule, and merge class name from the table to the point features at run-time, with the FeatureMerger or DatabaseJointer.

 

e.g.

 

codeclass (point type)asupermarketsbsupermarketscsports centredsports centre...
sunsilk11
Rank
Badge +6

@takashI

I don't have an attribute which denote supermarket etc but instead need to apply query like classification code in('a','b',...) to get all supermarket point features.

I'll look into using the JSON/List options

Thank you

Ok. This should work. I could probably use AttributeCreator with conditional statements to create new attribute.

 

I will try this out tomorrow.

 

 

sunsilk11
Rank
Badge +6

@takashI

I don't have an attribute which denote supermarket etc but instead need to apply query like classification code in('a','b',...) to get all supermarket point features.

I'll look into using the JSON/List options

Thank you

@takashi

 

I tried using AttributeValueMapper but encountered couple of issues:

 

1. It requires single source attribute

 

My queries for obtaining different counts are based on different fields

 

2. The aggregator seems to take a long time. My point layer contains 5 million records. FME optimizing memory message appears quite a few times.

 

 

 

 

takashi
Rank
Userlevel 2
Badge +17
@takashi

 

I tried using AttributeValueMapper but encountered couple of issues:

 

1. It requires single source attribute

 

My queries for obtaining different counts are based on different fields

 

2. The aggregator seems to take a long time. My point layer contains 5 million records. FME optimizing memory message appears quite a few times.

 

 

 

 

Yes, the AttributeValueMapper requires single source attribute. Do you mean that the classification code values ('a', 'b', ...) are stored into some different fields in a single feature separately? I'm unclear what schema of the source features is.

 

 

sunsilk11
Rank
Badge +6

@takashI

I don't have an attribute which denote supermarket etc but instead need to apply query like classification code in('a','b',...) to get all supermarket point features.

I'll look into using the JSON/List options

Thank you

Hi @takashi

 

No classification code for getting count of supermarkets, sport center is from one attribute.

 

Other counts e.g 'commercial property" I need are based on querying a different field in my case field Category.

 

 

 

takashi
Rank
Userlevel 2
Badge +17

@takashI

I don't have an attribute which denote supermarket etc but instead need to apply query like classification code in('a','b',...) to get all supermarket point features.

I'll look into using the JSON/List options

Thank you

If each point feature has an attribute that stores a classification code ("a", "b", "c", ... etc.) and you can create another attribute (called "_point_type" in my example) that stores a class name ("markets", "sports centre", "commercial property", ... etc.) corresponding to the code, then the workflow example I've posted should work as expected, except the performance issue.

 

 

sunsilk11
Rank
Badge +6

@takashI

I don't have an attribute which denote supermarket etc but instead need to apply query like classification code in('a','b',...) to get all supermarket point features.

I'll look into using the JSON/List options

Thank you

@takashi

 

I made some progress on this today. However I am a bit unclear on the class/code attributeValueMapper and aggregate parameters. Can you please clarify

 

My point fc is similar to attached

 

fme-test.png

 

 

takashi
Rank
Userlevel 2
Badge +17
@takashi

 

I made some progress on this today. However I am a bit unclear on the class/code attributeValueMapper and aggregate parameters. Can you please clarify

 

My point fc is similar to attached

 

fme-test.png

 

 

Your point features have "Description" attribute. I think it's the "_point_type" in my example. Isn't it?

 

 

sunsilk11
Rank
Badge +6

@takashI

I don't have an attribute which denote supermarket etc but instead need to apply query like classification code in('a','b',...) to get all supermarket point features.

I'll look into using the JSON/List options

Thank you

I think this is not working for me as the point fc has records with a large number of null values for code and description. This is throwing following message.

 

Attribute name is empty for value expression '@Value(POINT_TYPE)'. Ignoring expression since unable to create a blank attribute name

 

 

Thank you for you help so far

 

takashi
Rank
Userlevel 2
Badge +17
I think this is not working for me as the point fc has records with a large number of null values for code and description. This is throwing following message.

 

Attribute name is empty for value expression '@Value(POINT_TYPE)'. Ignoring expression since unable to create a blank attribute name

 

 

Thank you for you help so far

 

Which type should the points that don't have code/description be classified? Supermarkets? Sport Center?

 

 

takashi
Rank
Userlevel 2
Badge +17
I think this is not working for me as the point fc has records with a large number of null values for code and description. This is throwing following message.

 

Attribute name is empty for value expression '@Value(POINT_TYPE)'. Ignoring expression since unable to create a blank attribute name

 

 

Thank you for you help so far

 

Where does the 'POINT_TYPE' in the message come from? Did you rename "Description" to "POINT_TYPE"?

 

If you filtered out the points that don't have code/description, would the workflow work as expected?

 

 

sunsilk11
Rank
Badge +6

@takashI

I don't have an attribute which denote supermarket etc but instead need to apply query like classification code in('a','b',...) to get all supermarket point features.

I'll look into using the JSON/List options

Thank you

Yes, I filtered the records, and it does seem to work now.

 

I will now need to look at the performance.

 

Thanks

 

sunsilk11
Rank
Badge +6

@takashI

I don't have an attribute which denote supermarket etc but instead need to apply query like classification code in('a','b',...) to get all supermarket point features.

I'll look into using the JSON/List options

Thank you

@takashi

 

Hi, I am trying to see if the list/json option improves on the performance for my workspace.

 

I have a result with PolygonID, and json result as {"supermarkets":"3","school":"10", "":"120"}

 

Any suggestions how this can then be merged back to the original polygon feature class to populate the counts fields?

 

takashi
Rank
Userlevel 2
Badge +17

@takashI

I don't have an attribute which denote supermarket etc but instead need to apply query like classification code in('a','b',...) to get all supermarket point features. 

I'll look into using the JSON/List options

Thank you

I have already suggest a complete solution. If you have adopted other approach than this, please clarify what workflow you have created.

 

[re-post]

 

0684Q00000ArMioQAF.png

JSONTemplater | Root Template

 

{|
    let $count := fme:get-list-attribute("_histogram{}.count")
    for $type at $i in fme:get-list-attribute("_histogram{}.value")
    return {$type : $count[$i]}
|}

 

 

sunsilk11
Rank
Badge +6

@takashI

I don't have an attribute which denote supermarket etc but instead need to apply query like classification code in('a','b',...) to get all supermarket point features.

I'll look into using the JSON/List options

Thank you

Hi, I got this working now by using JsonExtractor to extract the count for supermarkets etc.. and join to polygon fc using a FeatureMerger.

 

However for polygons with no supermarkets etc.., the output fc is populated with Null, instead of zero.

 

 

sunsilk11
Rank
Badge +6

@takashI

I don't have an attribute which denote supermarket etc but instead need to apply query like classification code in('a','b',...) to get all supermarket point features.

I'll look into using the JSON/List options

Thank you

I am using the same solution, except added a JsonExtractor and FeatureMerger at the end.

 

I am setting my one of my target attribute in the JsonExtractor to json["Supermarket"], which work when this exist in the json document, otherwise the count stays as Null.

 

 

I would like to have a conditional statement as

 

int(json["Supermarket"]) > 0?json["Supermarket"]:0

 

 

fme-counts.png

 

 

takashi
Rank
Userlevel 2
Badge +17

@takashI

I don't have an attribute which denote supermarket etc but instead need to apply query like classification code in('a','b',...) to get all supermarket point features.

I'll look into using the JSON/List options

Thank you

You can use the NullAttributeMapper to replace <null>, <missing>, <empty> with a desired value.

 

However, I don't think you need to use the JSONExtractor. The JSONFlattener has already extracted count attributes for each point class (supermarket, sport center, etc.). You can just expose the attribute names via the Attributes to Expose parameter in the JSONFlattener.

 

takashi
Rank
Userlevel 2
Badge +17
You can use the NullAttributeMapper to replace <null>, <missing>, <empty> with a desired value.

 

However, I don't think you need to use the JSONExtractor. The JSONFlattener has already extracted count attributes for each point class (supermarket, sport center, etc.). You can just expose the attribute names via the Attributes to Expose parameter in the JSONFlattener.

 

Also, the features are from the Area port of the PoinOnAreaOverlayer unlike the first solution with the Aggregator, so I don't think you need to merge the resulting attributes back to the original areas.

 

 

sunsilk11
Rank
Badge +6

@takashI

I don't have an attribute which denote supermarket etc but instead need to apply query like classification code in('a','b',...) to get all supermarket point features.

I'll look into using the JSON/List options

Thank you

@takashi

 

This works perfectly. Thanks.

 

 

However the result has uncovered another issue, as some supermarkets and schools are being counted multiple times, as there are multiple point features denoting same supermarket or school for example. Each has a unique ID, but have a name field denoting the name of the supermarket.

 

I would need to count the below as 1, rather than 3

 

 

e.g.

 

 

ID Category Name Description

 

1 Supermarket Costco Costco Tyre Department

 

2 Supermarket Costco Costco Fuel

 

3 Supermarket Costco Costco Warehouse

Reply


Community Stats

28,709
Posts
109,484
Replies
36,556
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.