need Clipper to work in multicore/parallel setu but it is not; tried 2 diff ways

Question

Hi, I use parallel (//) processing in other workspaces successfully but this is the first time with a Clipper. The tips offered in article: FME Parallel Processing: Tips and tricks for generating groups | Safe Software do not seem to apply, or at least I think they don’t. I am trying to clip an OBJ of ~12720000 triangles with ~93K polygons of diiferent sizes. The OBJ has no attributes, just geometry.

1st try: link the same modcount (4 groups) to both Candidate and Clipper inputs of custom xformer containing only a Clipper. The data passess through it without any clipping. The Clipper inside is set as “multiple clippers”; Clippers do not arrive first; Reject only parts, merge-clipper, and the rest is set with defaults.

2nd try: cloning the OBJ as explained in article but it slow and eventually crashs as it runs out of memory. I sort of expected it but tried it anyway.

I don’t see how I can use the tiling method here. Am I missing anything else? or maybe there is an update on setting up the clipper to use // processing without workarounds that I am not aware of?

Thanks in advance,

using fme 2022.0.0.1

icon

Best answer by lmo_rott 29 March 2024, 16:33

View original

lmo_rott · Accepted Answer

@redgeographics thanks for responding and trying to help. Yes, both have the same CRS.

I ended up using a third dataset which allowed me to clip the TIN into large sections while adding to each section a “region” attribute to the othewise purely geometry TIN. This is similar to the “tiling” approach. Luckily this region attribute is also available for the clippers that I originally wanted to use. So I end up clipping the TIN 2x to then use the // setup when grouping by “regions”. Not very efficient when using only one region but at least it speeds up the processing as originally intended.

Interesting that the first approach works with the draper but not with the clipper.

redgeographics · Answer

Option 1 should, in theory, be the way to go. You do have to make sure that a Candidate from one group will only be clipped by Clippers from the same group. With those numbers I would expect to see some clipping occur even if the 4 groups are assigned randomly.

Wild hunch: are your Candidates and Clippers in the same coordinate system?

need Clipper to work in multicore/parallel setu but it is not; tried 2 diff ways

2 replies

Reply

Community Stats

Reply

Community Stats

Sign up

An FME Account is required to contribute

Login to the community

An FME Account is required to contribute

Scanning file for viruses.

This file cannot be downloaded