Solved

How to loop through records from two datasets based on matching values

  • 17 December 2016
  • 6 replies
  • 9 views
damian6105
Rank
Badge

I have 2 spatial datasets - FAUNA_MASTER and FAUNA_SAMPLE. Both datasets contain points (many thousand) representing the locations of a variety of animal species (Species A, Species B, Species C, ... Species Z). I want to create an FME script that identifies all records, for each species in FAUNA_SAMPLE that occur within 10 kilometers of all records, for the same corrosponding species in FAUNA_MASTER.

The transformers to use I think are straight forward, I use Bufferer to create a 10km buffer around points in FAUNA_MASTER, then use PointOnAreaOverlayer and a Tester transformer to identify which points in FAUNA_SAMPLE points occur within that buffer. What I don't know how to do is iterate or loop the records for each species through those transformers.

I'm relatively new to FME. I've tried my best to understand the documentation to create a looping custom transformer and apply it to my situation, but without success. If someone could walk me through it please (with baby steps) I would really appreciate it.

icon

Best answer by erik_jan 17 December 2016, 23:42

View original

6 replies

itay
Rank
Badge +16

Hi @damian6105, I don't think a looping custom transformer is necessary, I would use the PointOnPointOverlayer transformer with a group by (species attribute) with a tolerance of 10 kilometer.

This way points with the same value in the species attribute, from both inputs will be related and the attributes will be merged.

Hope this helps.

erik_jan
Rank
Userlevel 2
Badge +16

Another option is using the NeighborFinder transformer, group by Species, set maximum distance 10 (km) or 10000 (m) depending on the coordinate system and use a list to catch all occurrences in the other dataset.

See for more info: NeighborFinder

fmelizard
Rank
Userlevel 4
Badge +13

Another option is using the NeighborFinder transformer, group by Species, set maximum distance 10 (km) or 10000 (m) depending on the coordinate system and use a list to catch all occurrences in the other dataset.

See for more info: NeighborFinder

I like Erik's suggestion above. I think the key point is to understand how "lists" in FME prevent the need to loop. If an operation produces a 1 to Many type of result, often in FME we aggregate the "many" into a "list" of attributes on the "one". Then you can use a ListExploder to get back the "many" as individual features that you can then further test/modify/reroute as necessary.

 

 

I put together a small example showing this principle using the PointOnAreaOverlayer as the list maker:

 

 

pointonareaexample.fmwt

 

 

damian6105
Rank
Badge

Another option is using the NeighborFinder transformer, group by Species, set maximum distance 10 (km) or 10000 (m) depending on the coordinate system and use a list to catch all occurrences in the other dataset.

See for more info: NeighborFinder

Thanks @erik_jan for your response, as I said I'm relatively new to FME so hoping you can walk me through the NeighborFinder. So far these are my parameters:

 

BASE: FAUNA_SAMPLE

 

CANDIDATE: FAUNA_MASTER

 

Group By: SPECIES

 

Number of Neighbours to Find: 1 (presumably once it finds one other record of the same species within the 'Maximum Distance' it doesn't need to search anymore)

 

Maximum Distance: 10000 (i.e. 10 km)

 

Close Candidate List Name: _species

 

Then this is where I get stuck. I know I need to 'expose my elements', but which elements am I meant to expose? And how many? I've tried several variations without sucess. Any further assistance would be greatly appreciated.

 

 

itay
Rank
Badge +16

Thanks @erik_jan for your response, as I said I'm relatively new to FME so hoping you can walk me through the NeighborFinder. So far these are my parameters:

 

BASE: FAUNA_SAMPLE

 

CANDIDATE: FAUNA_MASTER

 

Group By: SPECIES

 

Number of Neighbours to Find: 1 (presumably once it finds one other record of the same species within the 'Maximum Distance' it doesn't need to search anymore)

 

Maximum Distance: 10000 (i.e. 10 km)

 

Close Candidate List Name: _species

 

Then this is where I get stuck. I know I need to 'expose my elements', but which elements am I meant to expose? And how many? I've tried several variations without sucess. Any further assistance would be greatly appreciated.

 

 

Hi @damian6105, Have a look at the example provided by @daleatsafe where a list is generated and exploded ( so that it can be used)

 

damian6105
Rank
Badge

Another option is using the NeighborFinder transformer, group by Species, set maximum distance 10 (km) or 10000 (m) depending on the coordinate system and use a list to catch all occurrences in the other dataset.

See for more info: NeighborFinder

Ahh, thanks @daleatsafe , @erik_jan, I just noticed the example posted by daleatsafe after I submitted my comment. Followed his list example using NeighbourFinder and worked a treat. Cheers.

Reply


Community Stats

28,731
Posts
109,530
Replies
36,567
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.