Solved

Dissolver accumulation

  • 12 February 2019
  • 6 replies
  • 3 views

Badge +8

Hello!

I'm using a dissolver before a pointonareaoverlayer and I'm trying to figure out how to be able to pass the areas from the dissolver to the pointonareaoverlayer before the points, so I can improve performance. I have it set up so it should read in all the records for the dissolver before the points.

icon

Best answer by fmelizard 14 February 2019, 05:07

View original

6 replies

Badge +2

I'm afraid you can't. You can't use the Dissolve and then ensure the dissolved features enter the PointOnAreaOverlayer first so you can take advantage of Areas First. In some cases you can take advantage of the Feature ordering using the Creator transformers (Workspaces Built to Order) introduced in 2016. But the addition of the Dissolver breaks this pattern.

Depending on the relative sizes of the datasets, points vs dissolved areas, it might be worthwhile splitting the workflow into two parts. Dissolve and write the resulting areas to a staging database or format (i..e Geopakage or FFS) then re-read the dissolved areas and the points to take advantage of PointOnAreaOverlayer - Areas First

If you can, I would try running your workflow in FME 2019. We have done a lot to improve both the performance and the results of the dissolve. Our testing shows that the 2019 Dissolver is between 2 to 76 times faster, depending on the input data.

Userlevel 3
Badge +13

This was a fascinating one and I thought it worth a think. Check the below workflow @runneals -- it has not been tested but I believe it should do as you want. The key trick is to use the Sampler to get us the LAST dissolved area AFTER that area has found its way into the PointOnAreaOverlayer. We ensure this by noting the port output order -- you can right click on a port to adjust these, but in general, whatever you connect first, goes there first, and 2nd, goes there second. So if we can get the last area this way, we then can use that as a trigger / initiator feature on the FeatureReader to pull our points out. Giddy up, it should just work. Let us know if it does.

Badge +2

This was a fascinating one and I thought it worth a think. Check the below workflow @runneals -- it has not been tested but I believe it should do as you want. The key trick is to use the Sampler to get us the LAST dissolved area AFTER that area has found its way into the PointOnAreaOverlayer. We ensure this by noting the port output order -- you can right click on a port to adjust these, but in general, whatever you connect first, goes there first, and 2nd, goes there second. So if we can get the last area this way, we then can use that as a trigger / initiator feature on the FeatureReader to pull our points out. Giddy up, it should just work. Let us know if it does.

@daleatsafe 's suggestion to use Sampler with:

Sampling Rate (N): 1

 

Sampling Type: Last N Features

solves the problem and enables the feature ordering you need to take advantage of PointOnAreaOverlayer's Areas First. I've attached an example workspace (2018.1): featureordering.zip

Badge +3

Statisticscalculator and have the summary trigger the featurereader?

Badge +8

This was a fascinating one and I thought it worth a think. Check the below workflow @runneals -- it has not been tested but I believe it should do as you want. The key trick is to use the Sampler to get us the LAST dissolved area AFTER that area has found its way into the PointOnAreaOverlayer. We ensure this by noting the port output order -- you can right click on a port to adjust these, but in general, whatever you connect first, goes there first, and 2nd, goes there second. So if we can get the last area this way, we then can use that as a trigger / initiator feature on the FeatureReader to pull our points out. Giddy up, it should just work. Let us know if it does.

Works like a charm! I'll test out 2019 and compare it. Thanks for the suggestion!

I didn't realize that you could move them up and down by right clicking on them... You learn something new everyday!

Badge +8

Statisticscalculator and have the summary trigger the featurereader?

I think those would work as well, but it seems like they would be more intensive than the sampler as they do statistics (so it would be a transformer that accumulates the data and performs calculations on it) so I would think it would negatively impact performance (especially when dealing with millions of records.

Reply