Solved

Extract Polygons as per Grid


Badge +1

I am sing FME 2016 and have polygons dataset that are spread in an irregular fashion. I want to create a fishnet(grid) wherein the geometry is as per the density of the polygons. So, first of all, I have to convert the polygons to points, so that the original polygon file is not occupied by 2 different grids and then create the tiles thing and save the output in multiple polygon gdbs (outputs as per the tile). it would be really amazing if the number of polygons in one grid could be controlled.

Thanks.

icon

Best answer by takashi 16 May 2017, 10:54

View original

16 replies

Badge

So use BoundingBoxReplacer for a set of polygons, then tile it with tiler, do a CenterPointReplacer for each polygon and test it against the tiles with PointOnAreaOverlayer. I think this would do what you want.

Regarding controlling the number of polygons in one grid... not sure what you mean. Maybe you want to make an iteration and divide the tiles to sub-tiles untill you reach a certain number of polygons within a tile?

Badge +1

Thanks for your reply Jaro. Actually, am having an issue to distribute the polygons homogeneously. Finding some solution, I had seen a post from @takashi

 

wherein he had created a workbench for the same. I am attaching a sample data that might help you.

What I actually want:

Split the data in grid structures. If you look at the data, there are areas where hardly polygons present whereas at some locations very tiny water polygons are there. So, the general division of polygons (first converting the polygons to points (since 1 polygon might occur in multiple grids) and then again to the original shape(polygon))). And if, I create general grids (and I want all the water polygons as per grid leading as the number of grids equal to the output gdbs), there might be cases wherein the gdbs are empty, since there would be no water polygons. Again, we had to find the empty gdb and remove it before the data is ingested into the further process.

Looking into all these issues, I had thought of dividing the polygons in the manner of their density (like 2000 polygons in 1 gdb). This will have- 1) equally distributed count of polygons 2) I would be able to set how many number of polygons I want in each grid 3) No empty gdbs.

Thanks.

Userlevel 2
Badge +17

Hi @hellblazer, sorry, I cannot grasp the main point of your question. If you are looking for a way to create fishnet grid, the 2DGridAccumulator might help you.

Badge +1

Hi @hellblazer, sorry, I cannot grasp the main point of your question. If you are looking for a way to create fishnet grid, the 2DGridAccumulator might help you.

If you have a look at the below image, the water polygons are heterogeneous. the are covered under circle 1 consists of little or no polygons whereas the second circle is flooded with polygons. What I want: Divide the polygons in equal number of polygons per file (lets say 200 polygons in 1 gdb. Once 200 are filled, the next count of 200 goes to other gdb and so on...). Now, according to digitization of my data, the polygon1 is the one at top left, the other one might be bottom right one and randomly distributed like this. I want a true distribution of data spatially as per the count of my wish. This led me to the distribution of data by creating grids according to the count/polygon density. I hope I am understandable. Thanks for your time.

 

 

 

Userlevel 2
Badge +17

Hi @hellblazer, sorry, I cannot grasp the main point of your question. If you are looking for a way to create fishnet grid, the 2DGridAccumulator might help you.

I understood the requirement, but it's hard (almost impossible) to create grid cells (rectangles) each of which contains exactly same number of polygons, since the distribution of the polygons is irregular.

 

However, if the condition about the number of polygons within a cell was given as the maximum number per cell (e.g. 200 or less), it would be possible by dividing bounding box recursively, as @jaro mentioned.

 

I remember that I definitely posted a similar solution maybe using Python before, but I was not able to find out the thread.
Badge +1

So use BoundingBoxReplacer for a set of polygons, then tile it with tiler, do a CenterPointReplacer for each polygon and test it against the tiles with PointOnAreaOverlayer. I think this would do what you want.

Regarding controlling the number of polygons in one grid... not sure what you mean. Maybe you want to make an iteration and divide the tiles to sub-tiles untill you reach a certain number of polygons within a tile?

Thanks for your suggestion @jaro. My motive is to split the data spatially. Might not be equal number of polygons or might not be the grids to be perfectly rectangular. It would be great if I can control the polygons count.

 

Badge +1

Hi @hellblazer, sorry, I cannot grasp the main point of your question. If you are looking for a way to create fishnet grid, the 2DGridAccumulator might help you.

That's okay. It might not be perfectly rectangular. All I want to split the data spatially and if possible, control the counts of polygon per file.

 

Userlevel 2
Badge +17

Hi @hellblazer, sorry, I cannot grasp the main point of your question. If you are looking for a way to create fishnet grid, the 2DGridAccumulator might help you.

Something like this?

 

  • Assuming the point is a representative point of the polygon (center point, center of mass, or inside point).
  • The number of points per cell is approximately identical (allowing +/- 1 error).

Badge +1

Hi @hellblazer, sorry, I cannot grasp the main point of your question. If you are looking for a way to create fishnet grid, the 2DGridAccumulator might help you.

Just perfect as I want. Can we specify the number of points(polygons) as per our wish?

 

 

Userlevel 2
Badge +17

You can specify the maximum N (number of points per cell). If the number of source points was equal to N x 2^m (2 raised to the power of m; m > 0), every cell will contain exactly N points. Otherwise, cells contain almost identical number (equal to or less than N) of points.

See this workspace example containing a PythonCaller: irregular-grid.fmw (FME 2017.0.1)

# PythonCaller Script Exampleimport fmeobjectsclass FeatureProcessor(object):    def __init__(self):        self.maxNumPointsPerCell = 20 # Maximum number of points per cell        self.features = [] # list of (x, y, feature)            def input(self, feature):        if feature.getGeometryType() == fmeobjects.FME_GEOM_POINT:            coord = feature.getCoordinate(0)            self.features.append((coord[0], coord[1], feature))            def close(self):        if self.features:            self.cell_id = 0            xs = sorted([x for x, _, _ in self.features])            ys = sorted([y for _, y, _ in self.features])            self.distribute(self.features, xs[0], ys[0], xs[-1], ys[-1])            def distribute(self, features, xmin, ymin, xmax, ymax):        if features:            if len(features) <= self.maxNumPointsPerCell:                for _, _, feat in features:                    feat.setAttribute('_cell_id', self.cell_id)                    self.pyoutput(feat)                coords = [(xmin,ymin),(xmax,ymin),(xmax,ymax),(xmin,ymax),(xmin,ymin)]                cell = fmeobjects.FMEFeature()                cell.setGeometry(fmeobjects.FMEPolygon(fmeobjects.FMELine(coords)))                cell.setAttribute('_cell_id', self.cell_id)                cell.setAttribute('_count', len(features))                self.pyoutput(cell)                self.cell_id += 1            else:                n = len(features) // 2                if (xmax - xmin) < (ymax - ymin):                    features.sort(key=lambda f:f[1])                    ymid = (features[n-1][1] + features[n][1]) * 0.5                    self.distribute(features[:n], xmin, ymin, xmax, ymid)                    self.distribute(features[n:], xmin, ymid, xmax, ymax)                else:                    features.sort(key=lambda f:f[0])                    xmid = (features[n-1][0] + features[n][0]) * 0.5                    self.distribute(features[:n], xmin, ymin, xmid, ymax)                    self.distribute(features[n:], xmid, ymin, xmax, ymax)
Badge +1

Thanks a lot for the support Takashi. If I run the workbench, it simply runs and gives the desired output. But, if I use some sample data, the PythonCaller is not responding and giving a null output but a successful workbench. I have attached the data. samplepolygon.zip

I am not much of a programming person, but are there some changes that I need to do in python script?

Userlevel 2
Badge +17

Thanks a lot for the support Takashi. If I run the workbench, it simply runs and gives the desired output. But, if I use some sample data, the PythonCaller is not responding and giving a null output but a successful workbench. I have attached the data. samplepolygon.zip

I am not much of a programming person, but are there some changes that I need to do in python script?

Check these points.

 

  • The script accepts only point features. Make sure that you have transformed each polygon into its representative point with the CenterPointReplacer.
  • The polygons have a coordinate system, but the cell features created by the script does not have any coordinate system. If you are viewing them with FME Data Inspector having set Background Map option, the features may not be shown correctly. To resolve this, set an appropriate coordinate system to the cell features.
You can modify the maximum number of points per cell here.

 

# PythonCaller Script Example
import fmeobjects
class FeatureProcessor(object):
    def __init__(self):
        self.maxNumPointsPerCell = 50 # Maximum number of points per cell
        ...
Userlevel 2
Badge +17

Thanks a lot for the support Takashi. If I run the workbench, it simply runs and gives the desired output. But, if I use some sample data, the PythonCaller is not responding and giving a null output but a successful workbench. I have attached the data. samplepolygon.zip

I am not much of a programming person, but are there some changes that I need to do in python script?

This is the result from your sample data. Center point has been used as the representative point for each polygon. The maximum number of points per cell has been set to 64, but the resulting number of points per cell is 52 or 53 (< 64).

 

Userlevel 2
Badge +17

You can specify the maximum N (number of points per cell). If the number of source points was equal to N x 2^m (2 raised to the power of m; m > 0), every cell will contain exactly N points. Otherwise, cells contain almost identical number (equal to or less than N) of points.

See this workspace example containing a PythonCaller: irregular-grid.fmw (FME 2017.0.1)

# PythonCaller Script Exampleimport fmeobjectsclass FeatureProcessor(object):    def __init__(self):        self.maxNumPointsPerCell = 20 # Maximum number of points per cell        self.features = [] # list of (x, y, feature)            def input(self, feature):        if feature.getGeometryType() == fmeobjects.FME_GEOM_POINT:            coord = feature.getCoordinate(0)            self.features.append((coord[0], coord[1], feature))            def close(self):        if self.features:            self.cell_id = 0            xs = sorted([x for x, _, _ in self.features])            ys = sorted([y for _, y, _ in self.features])            self.distribute(self.features, xs[0], ys[0], xs[-1], ys[-1])            def distribute(self, features, xmin, ymin, xmax, ymax):        if features:            if len(features) <= self.maxNumPointsPerCell:                for _, _, feat in features:                    feat.setAttribute('_cell_id', self.cell_id)                    self.pyoutput(feat)                coords = [(xmin,ymin),(xmax,ymin),(xmax,ymax),(xmin,ymax),(xmin,ymin)]                cell = fmeobjects.FMEFeature()                cell.setGeometry(fmeobjects.FMEPolygon(fmeobjects.FMELine(coords)))                cell.setAttribute('_cell_id', self.cell_id)                cell.setAttribute('_count', len(features))                self.pyoutput(cell)                self.cell_id += 1            else:                n = len(features) // 2                if (xmax - xmin) < (ymax - ymin):                    features.sort(key=lambda f:f[1])                    ymid = (features[n-1][1] + features[n][1]) * 0.5                    self.distribute(features[:n], xmin, ymin, xmax, ymid)                    self.distribute(features[n:], xmin, ymid, xmax, ymax)                else:                    features.sort(key=lambda f:f[0])                    xmid = (features[n-1][0] + features[n][0]) * 0.5                    self.distribute(features[:n], xmin, ymin, xmid, ymax)                    self.distribute(features[n:], xmid, ymin, xmax, ymax)
This workspace demonstrates how the script works using @hellblazer's sample data.

 

esrishape2none.fmwt (including a sample dataset, FME 2016.0)

 

Badge +1

Thanks a lot for the support Takashi. If I run the workbench, it simply runs and gives the desired output. But, if I use some sample data, the PythonCaller is not responding and giving a null output but a successful workbench. I have attached the data. samplepolygon.zip

I am not much of a programming person, but are there some changes that I need to do in python script?

8099-irregular-grid.zip

 

I tried to amend the workbench that you have sent @takashi (in the attachment), but sorry, I am only getting the tiles and not the original data. Could you please help me out here.

 

Userlevel 2
Badge +17
8099-irregular-grid.zip

 

I tried to amend the workbench that you have sent @takashi (in the attachment), but sorry, I am only getting the tiles and not the original data. Could you please help me out here.

 

I cannot understand why you are using the Offsetters. The Offsetters in my first demo workspace just create test data (points) located randomly.

 

See the workspace example "esrishape2none.fmwt" in the comment I added later.

 

Reply