Question

Dynamically filter unique values? Attribute-based probability sampling?

  • 26 February 2015
  • 2 replies
  • 53 views

Hi all, 

 

 

I am relatively new to FME, it would be greatly appreciated if someone can shed some light on a couple of problems I can't seem to figure out.

 

 

1.

 

 

Is there a transformer similar to the attribute filter transformer but would enable me to route unique value to the output port dynamically? I understand that I can using the import value function to bring in the unique attributes values, but sometimes I would only like to only filter a portion (which could be about 50 unique values) of the total number of unique values.  Inputting the required attribute value manually is laborious and prone to mistakes. 

 

 

What I would like to achieve is a 2 step dynamic filter of features, each feature has a Local Government Area (LGA), and a Suburb (suburb is a subset of LGA) attribute. Depending of the first LGA filter (ideally a check box list), the second transformer would route filtered features with unique suburb attributes to an output port. Is this possible?

 

 

2.

 

 

I am currently trying to incorporate a randomised probability sampling component to my workspace based on an attribute. In essence, it tries to simplisticly simulate urban development for x number of years at a parcel level.

 

 

The parcel features in my dataset has a "development_likelihood" attribute that is calculated based on summary statistics of similar types of features from a different dataset. The "development_likelihood" attribute determines the percentage of features that gets randomly sampled. The sampled features will receive a new attribute with the year when it was selected/developed. The un-sampled features will be passed on to the next sampling process. The intent is the loop this process for x number of years and visualise which lots get developed year by year.

 

 

I have so far found that the randomizedSampler transformer on the FME store is close to what I want to achieve in terms of sampling features randomly, however the Percentage to Sample parameter must be set by a published parameter, hence not able to read the likelihood attribute from the passing features.

 

 

How would I be able to get the transformer to use the "likelihood" attribute as an input to determine the sampling amount for a group of features that have the same value? I think I will end up with roughly 100 unique values, which would be another problem if I need to create 100 of these transformers all with different likelihood values. Any suggestion to improve on my simplistic approach would be much appreciated.

 

 

Regards,

 

MC

2 replies

Userlevel 4
Hi,

 

 

for question no 1, look at the DuplicateRemover.

 

 

For no. 2, maybe it would be an idea to split your processing using the WorkspaceRunner. That way you could pass the "likelyhood" attribute as a parameter to the next workspace.

 

 

David
Badge +3
Hi,

 

 

There is a earlier post about Random Stratified Sampler (search on this)

 

I posted a workspace doing this. There are plenty statistics forums explaining how to do this. (different topic, same principle)

 

 

Maybe it is usefull to you(?)

Reply