Skip to main content
Solved

Extract Polygons as per Grid

parashari
Forum|alt.badge.img+2

I am sing FME 2016 and have polygons dataset that are spread in an irregular fashion. I want to create a fishnet(grid) wherein the geometry is as per the density of the polygons. So, first of all, I have to convert the polygons to points, so that the original polygon file is not occupied by 2 different grids and then create the tiles thing and save the output in multiple polygon gdbs (outputs as per the tile). it would be really amazing if the number of polygons in one grid could be controlled.

Thanks.

Best answer by takashi

You can specify the maximum N (number of points per cell). If the number of source points was equal to N x 2^m (2 raised to the power of m; m > 0), every cell will contain exactly N points. Otherwise, cells contain almost identical number (equal to or less than N) of points.

See this workspace example containing a PythonCaller: irregular-grid.fmw (FME 2017.0.1)

# PythonCaller Script Exampleimport fmeobjectsclass FeatureProcessor(object):    def __init__(self):        self.maxNumPointsPerCell = 20 # Maximum number of points per cell        self.features = [] # list of (x, y, feature)            def input(self, feature):        if feature.getGeometryType() == fmeobjects.FME_GEOM_POINT:            coord = feature.getCoordinate(0)            self.features.append((coord[0], coord[1], feature))            def close(self):        if self.features:            self.cell_id = 0            xs = sorted([x for x, _, _ in self.features])            ys = sorted([y for _, y, _ in self.features])            self.distribute(self.features, xs[0], ys[0], xs[-1], ys[-1])            def distribute(self, features, xmin, ymin, xmax, ymax):        if features:            if len(features) <= self.maxNumPointsPerCell:                for _, _, feat in features:                    feat.setAttribute('_cell_id', self.cell_id)                    self.pyoutput(feat)                coords = [(xmin,ymin),(xmax,ymin),(xmax,ymax),(xmin,ymax),(xmin,ymin)]                cell = fmeobjects.FMEFeature()                cell.setGeometry(fmeobjects.FMEPolygon(fmeobjects.FMELine(coords)))                cell.setAttribute('_cell_id', self.cell_id)                cell.setAttribute('_count', len(features))                self.pyoutput(cell)                self.cell_id += 1            else:                n = len(features) // 2                if (xmax - xmin) < (ymax - ymin):                    features.sort(key=lambda f:f[1])                    ymid = (features[n-1][1] + features[n][1]) * 0.5                    self.distribute(features[:n], xmin, ymin, xmax, ymid)                    self.distribute(features[n:], xmin, ymid, xmax, ymax)                else:                    features.sort(key=lambda f:f[0])                    xmid = (features[n-1][0] + features[n][0]) * 0.5                    self.distribute(features[:n], xmin, ymin, xmid, ymax)                    self.distribute(features[n:], xmid, ymin, xmax, ymax)
View original
Did this help you find an answer to your question?

16 replies

jaro
Contributor
Forum|alt.badge.img+1
  • jaroContributor
  • Contributor
  • May 15, 2017

So use BoundingBoxReplacer for a set of polygons, then tile it with tiler, do a CenterPointReplacer for each polygon and test it against the tiles with PointOnAreaOverlayer. I think this would do what you want.

Regarding controlling the number of polygons in one grid... not sure what you mean. Maybe you want to make an iteration and divide the tiles to sub-tiles untill you reach a certain number of polygons within a tile?

parashari
Forum|alt.badge.img+2
  • parashari
  • Author
  • May 16, 2017

Thanks for your reply Jaro. Actually, am having an issue to distribute the polygons homogeneously. Finding some solution, I had seen a post from @takashi

 

wherein he had created a workbench for the same. I am attaching a sample data that might help you.

What I actually want:

Split the data in grid structures. If you look at the data, there are areas where hardly polygons present whereas at some locations very tiny water polygons are there. So, the general division of polygons (first converting the polygons to points (since 1 polygon might occur in multiple grids) and then again to the original shape(polygon))). And if, I create general grids (and I want all the water polygons as per grid leading as the number of grids equal to the output gdbs), there might be cases wherein the gdbs are empty, since there would be no water polygons. Again, we had to find the empty gdb and remove it before the data is ingested into the further process.

Looking into all these issues, I had thought of dividing the polygons in the manner of their density (like 2000 polygons in 1 gdb). This will have- 1) equally distributed count of polygons 2) I would be able to set how many number of polygons I want in each grid 3) No empty gdbs.

Thanks.

takashi
Contributor
Forum|alt.badge.img+19
  • takashiContributor
  • Contributor
  • May 16, 2017

Hi @hellblazer, sorry, I cannot grasp the main point of your question. If you are looking for a way to create fishnet grid, the 2DGridAccumulator might help you.

parashari
Forum|alt.badge.img+2
  • parashari
  • Author
  • May 16, 2017
takashi wrote:

Hi @hellblazer, sorry, I cannot grasp the main point of your question. If you are looking for a way to create fishnet grid, the 2DGridAccumulator might help you.

If you have a look at the below image, the water polygons are heterogeneous. the are covered under circle 1 consists of little or no polygons whereas the second circle is flooded with polygons. What I want: Divide the polygons in equal number of polygons per file (lets say 200 polygons in 1 gdb. Once 200 are filled, the next count of 200 goes to other gdb and so on...). Now, according to digitization of my data, the polygon1 is the one at top left, the other one might be bottom right one and randomly distributed like this. I want a true distribution of data spatially as per the count of my wish. This led me to the distribution of data by creating grids according to the count/polygon density. I hope I am understandable. Thanks for your time.

 

 

 

takashi
Contributor
Forum|alt.badge.img+19
  • takashiContributor
  • Contributor
  • May 16, 2017
takashi wrote:

Hi @hellblazer, sorry, I cannot grasp the main point of your question. If you are looking for a way to create fishnet grid, the 2DGridAccumulator might help you.

I understood the requirement, but it's hard (almost impossible) to create grid cells (rectangles) each of which contains exactly same number of polygons, since the distribution of the polygons is irregular.

 

However, if the condition about the number of polygons within a cell was given as the maximum number per cell (e.g. 200 or less), it would be possible by dividing bounding box recursively, as @jaro mentioned.

 

I remember that I definitely posted a similar solution maybe using Python before, but I was not able to find out the thread.
parashari
Forum|alt.badge.img+2
  • parashari
  • Author
  • May 16, 2017
jaro wrote:

So use BoundingBoxReplacer for a set of polygons, then tile it with tiler, do a CenterPointReplacer for each polygon and test it against the tiles with PointOnAreaOverlayer. I think this would do what you want.

Regarding controlling the number of polygons in one grid... not sure what you mean. Maybe you want to make an iteration and divide the tiles to sub-tiles untill you reach a certain number of polygons within a tile?

Thanks for your suggestion @jaro. My motive is to split the data spatially. Might not be equal number of polygons or might not be the grids to be perfectly rectangular. It would be great if I can control the polygons count.

 

parashari
Forum|alt.badge.img+2
  • parashari
  • Author
  • May 16, 2017
takashi wrote:

Hi @hellblazer, sorry, I cannot grasp the main point of your question. If you are looking for a way to create fishnet grid, the 2DGridAccumulator might help you.

That's okay. It might not be perfectly rectangular. All I want to split the data spatially and if possible, control the counts of polygon per file.

 

takashi
Contributor
Forum|alt.badge.img+19
  • takashiContributor
  • Contributor
  • May 16, 2017
takashi wrote:

Hi @hellblazer, sorry, I cannot grasp the main point of your question. If you are looking for a way to create fishnet grid, the 2DGridAccumulator might help you.

Something like this?

 

  • Assuming the point is a representative point of the polygon (center point, center of mass, or inside point).
  • The number of points per cell is approximately identical (allowing +/- 1 error).

parashari
Forum|alt.badge.img+2
  • parashari
  • Author
  • May 16, 2017
takashi wrote:

Hi @hellblazer, sorry, I cannot grasp the main point of your question. If you are looking for a way to create fishnet grid, the 2DGridAccumulator might help you.

Just perfect as I want. Can we specify the number of points(polygons) as per our wish?

 

 

takashi
Contributor
Forum|alt.badge.img+19
  • takashiContributor
  • Contributor
  • Best Answer
  • May 16, 2017

You can specify the maximum N (number of points per cell). If the number of source points was equal to N x 2^m (2 raised to the power of m; m > 0), every cell will contain exactly N points. Otherwise, cells contain almost identical number (equal to or less than N) of points.

See this workspace example containing a PythonCaller: irregular-grid.fmw (FME 2017.0.1)

# PythonCaller Script Exampleimport fmeobjectsclass FeatureProcessor(object):    def __init__(self):        self.maxNumPointsPerCell = 20 # Maximum number of points per cell        self.features = [] # list of (x, y, feature)            def input(self, feature):        if feature.getGeometryType() == fmeobjects.FME_GEOM_POINT:            coord = feature.getCoordinate(0)            self.features.append((coord[0], coord[1], feature))            def close(self):        if self.features:            self.cell_id = 0            xs = sorted([x for x, _, _ in self.features])            ys = sorted([y for _, y, _ in self.features])            self.distribute(self.features, xs[0], ys[0], xs[-1], ys[-1])            def distribute(self, features, xmin, ymin, xmax, ymax):        if features:            if len(features) <= self.maxNumPointsPerCell:                for _, _, feat in features:                    feat.setAttribute('_cell_id', self.cell_id)                    self.pyoutput(feat)                coords = [(xmin,ymin),(xmax,ymin),(xmax,ymax),(xmin,ymax),(xmin,ymin)]                cell = fmeobjects.FMEFeature()                cell.setGeometry(fmeobjects.FMEPolygon(fmeobjects.FMELine(coords)))                cell.setAttribute('_cell_id', self.cell_id)                cell.setAttribute('_count', len(features))                self.pyoutput(cell)                self.cell_id += 1            else:                n = len(features) // 2                if (xmax - xmin) < (ymax - ymin):                    features.sort(key=lambda f:f[1])                    ymid = (features[n-1][1] + features[n][1]) * 0.5                    self.distribute(features[:n], xmin, ymin, xmax, ymid)                    self.distribute(features[n:], xmin, ymid, xmax, ymax)                else:                    features.sort(key=lambda f:f[0])                    xmid = (features[n-1][0] + features[n][0]) * 0.5                    self.distribute(features[:n], xmin, ymin, xmid, ymax)                    self.distribute(features[n:], xmid, ymin, xmax, ymax)
parashari
Forum|alt.badge.img+2
  • parashari
  • Author
  • May 16, 2017

Thanks a lot for the support Takashi. If I run the workbench, it simply runs and gives the desired output. But, if I use some sample data, the PythonCaller is not responding and giving a null output but a successful workbench. I have attached the data. samplepolygon.zip

I am not much of a programming person, but are there some changes that I need to do in python script?

takashi
Contributor
Forum|alt.badge.img+19
  • takashiContributor
  • Contributor
  • May 16, 2017
parashari wrote:

Thanks a lot for the support Takashi. If I run the workbench, it simply runs and gives the desired output. But, if I use some sample data, the PythonCaller is not responding and giving a null output but a successful workbench. I have attached the data. samplepolygon.zip

I am not much of a programming person, but are there some changes that I need to do in python script?

Check these points.

 

  • The script accepts only point features. Make sure that you have transformed each polygon into its representative point with the CenterPointReplacer.
  • The polygons have a coordinate system, but the cell features created by the script does not have any coordinate system. If you are viewing them with FME Data Inspector having set Background Map option, the features may not be shown correctly. To resolve this, set an appropriate coordinate system to the cell features.
You can modify the maximum number of points per cell here.

 

# PythonCaller Script Example
import fmeobjects
class FeatureProcessor(object):
    def __init__(self):
        self.maxNumPointsPerCell = 50 # Maximum number of points per cell
        ...
takashi
Contributor
Forum|alt.badge.img+19
  • takashiContributor
  • Contributor
  • May 16, 2017
parashari wrote:

Thanks a lot for the support Takashi. If I run the workbench, it simply runs and gives the desired output. But, if I use some sample data, the PythonCaller is not responding and giving a null output but a successful workbench. I have attached the data. samplepolygon.zip

I am not much of a programming person, but are there some changes that I need to do in python script?

This is the result from your sample data. Center point has been used as the representative point for each polygon. The maximum number of points per cell has been set to 64, but the resulting number of points per cell is 52 or 53 (< 64).

 

takashi
Contributor
Forum|alt.badge.img+19
  • takashiContributor
  • Contributor
  • May 17, 2017
takashi wrote:

You can specify the maximum N (number of points per cell). If the number of source points was equal to N x 2^m (2 raised to the power of m; m > 0), every cell will contain exactly N points. Otherwise, cells contain almost identical number (equal to or less than N) of points.

See this workspace example containing a PythonCaller: irregular-grid.fmw (FME 2017.0.1)

# PythonCaller Script Exampleimport fmeobjectsclass FeatureProcessor(object):    def __init__(self):        self.maxNumPointsPerCell = 20 # Maximum number of points per cell        self.features = [] # list of (x, y, feature)            def input(self, feature):        if feature.getGeometryType() == fmeobjects.FME_GEOM_POINT:            coord = feature.getCoordinate(0)            self.features.append((coord[0], coord[1], feature))            def close(self):        if self.features:            self.cell_id = 0            xs = sorted([x for x, _, _ in self.features])            ys = sorted([y for _, y, _ in self.features])            self.distribute(self.features, xs[0], ys[0], xs[-1], ys[-1])            def distribute(self, features, xmin, ymin, xmax, ymax):        if features:            if len(features) <= self.maxNumPointsPerCell:                for _, _, feat in features:                    feat.setAttribute('_cell_id', self.cell_id)                    self.pyoutput(feat)                coords = [(xmin,ymin),(xmax,ymin),(xmax,ymax),(xmin,ymax),(xmin,ymin)]                cell = fmeobjects.FMEFeature()                cell.setGeometry(fmeobjects.FMEPolygon(fmeobjects.FMELine(coords)))                cell.setAttribute('_cell_id', self.cell_id)                cell.setAttribute('_count', len(features))                self.pyoutput(cell)                self.cell_id += 1            else:                n = len(features) // 2                if (xmax - xmin) < (ymax - ymin):                    features.sort(key=lambda f:f[1])                    ymid = (features[n-1][1] + features[n][1]) * 0.5                    self.distribute(features[:n], xmin, ymin, xmax, ymid)                    self.distribute(features[n:], xmin, ymid, xmax, ymax)                else:                    features.sort(key=lambda f:f[0])                    xmid = (features[n-1][0] + features[n][0]) * 0.5                    self.distribute(features[:n], xmin, ymin, xmid, ymax)                    self.distribute(features[n:], xmid, ymin, xmax, ymax)
This workspace demonstrates how the script works using @hellblazer's sample data.

 

esrishape2none.fmwt (including a sample dataset, FME 2016.0)

 

parashari
Forum|alt.badge.img+2
  • parashari
  • Author
  • May 19, 2017
parashari wrote:

Thanks a lot for the support Takashi. If I run the workbench, it simply runs and gives the desired output. But, if I use some sample data, the PythonCaller is not responding and giving a null output but a successful workbench. I have attached the data. samplepolygon.zip

I am not much of a programming person, but are there some changes that I need to do in python script?

8099-irregular-grid.zip

 

I tried to amend the workbench that you have sent @takashi (in the attachment), but sorry, I am only getting the tiles and not the original data. Could you please help me out here.

 

takashi
Contributor
Forum|alt.badge.img+19
  • takashiContributor
  • Contributor
  • May 19, 2017
parashari wrote:
8099-irregular-grid.zip

 

I tried to amend the workbench that you have sent @takashi (in the attachment), but sorry, I am only getting the tiles and not the original data. Could you please help me out here.

 

I cannot understand why you are using the Offsetters. The Offsetters in my first demo workspace just create test data (points) located randomly.

 

See the workspace example "esrishape2none.fmwt" in the comment I added later.

 

Reply


Helpful Members This Week

Community Stats

30,890
Posts
117,076
Replies
38,698
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.