I have a very large dataset (around 1 Million polygons) and I want to dissolve touching polygons so that I can give them one unique ID to then conduct a surface on surface overlayer after a spatial filter based on the group ID of the dissolved polygons. My problem is that after the Dissolve the remnants part with the line features gets very big and exceeds my memory. Is there another more memory-efficient way to handle this?
@droiai How do you know that the <Remnants> output is exceeding your memory? What log message do you get? If you are using feature caching, perhaps try switching that off, or wrap the Dissolver in a bookmark and collapse the bookmark - feature caching is switched off inside collapsed bookmarks)
@droiai How do you know that the <Remnants> output is exceeding your memory? What log message do you get? If you are using feature caching, perhaps try switching that off, or wrap the Dissolver in a bookmark and collapse the bookmark - feature caching is switched off inside collapsed bookmarks)
Thanks for the answer - I thought that it was the remnants part because when caching was activated it did not come to the "Area" part and my memory was fastly decreasing when it processed the "Dissolving" part. I did not know that I could disable feature caching - thanks for the information. But unfortunately that did not solve the problem, I still get the error of insufficient memory.
What I don't understand is that it reads all my Shapefiles and the Wall-Surfaces (millions) without allocating much in my memory but as soon as it comes to the dissolve part it allocates a lot of memory.
Thanks for the answer - I thought that it was the remnants part because when caching was activated it did not come to the "Area" part and my memory was fastly decreasing when it processed the "Dissolving" part. I did not know that I could disable feature caching - thanks for the information. But unfortunately that did not solve the problem, I still get the error of insufficient memory.
What I don't understand is that it reads all my Shapefiles and the Wall-Surfaces (millions) without allocating much in my memory but as soon as it comes to the dissolve part it allocates a lot of memory.
Dissolving is a pretty hefty operation unfortunately. It sounds like you either need more memory/temp space -or- need to break this up into smaller groups of features.
Hi @droiai The error you got could be a problem that we need to address. Which build/release of FME are you using? 32 or 64 bit? Are you able to share the source data with us? Thanks