Question
is it possible to sort based on geography
I am building large datasets and am outputting to a geojson. Since I can't put in an index, I was wondering if it was possible to sort the feature order from east to west?
+15- Contributor
@gsteinmon
To sort objects from left to right you cabn use goniometry.
I fiddled for couple of hours (3-ish morel like) and found this one.
1. Create centerpoint for objects (or inside point).
2. Extract vertex.
3. And then create following value using the expression evaluator:
@Value(_y)-@atan(@Value(_x)/@Value(_y))*10000
Sort by this value and then use a count to create a attribute for order (else you be looking at largish numbers, counter makes it bit tidier)
The value 1000 is to garantee sufficient resolution, so each object has a unique value. (if u have zillions of objects you might need to increase this value)
The expression is for left to right, top down.
With little fidling you can reverse the order or make it right to left (in case you're like Asian...)
I made this years ago so people can find objects on maps easier.
haves fun,
+15- Contributor
gio wrote:
@gsteinmon
To sort objects from left to right you cabn use goniometry.
I fiddled for couple of hours (3-ish morel like) and found this one.
1. Create centerpoint for objects (or inside point).
2. Extract vertex.
3. And then create following value using the expression evaluator:
@Value(_y)-@atan(@Value(_x)/@Value(_y))*10000
Sort by this value and then use a count to create a attribute for order (else you be looking at largish numbers, counter makes it bit tidier)
The value 1000 is to garantee sufficient resolution, so each object has a unique value. (if u have zillions of objects you might need to increase this value)
The expression is for left to right, top down.
With little fidling you can reverse the order or make it right to left (in case you're like Asian...)
I made this years ago so people can find objects on maps easier.
haves fun,
+44- Safer
It depends on the definition of "east-west" but you don't technically need an InsidePoint- or CentrePoint- Replacer. You could just use the CoordinateExtractor to fetch the x/y of the first vertex in a polygon and then use the Sorter on that. Of course, where the first coordinate is in the feature could potentially affect the overall sort order.
+28- Contributor
mark2atsafe wrote:
It depends on the definition of "east-west" but you don't technically need an InsidePoint- or CentrePoint- Replacer. You could just use the CoordinateExtractor to fetch the x/y of the first vertex in a polygon and then use the Sorter on that. Of course, where the first coordinate is in the feature could potentially affect the overall sort order.
Alternatively use a BoundsExtractor to get the minimum/maximum coordinate.
+15- Contributor
As Mark said, the geometric shape of the objects as wel as their position and or possible overlaps does influence the order. You dont want to use points wich are say in 2 or more objects simultaneously.
This however can (mostly) be countered if u use a boundingbox accumulator grouped by object ID.
Then u use a AreOnAreoverlayer and tester to filter the BB parts wich do not overlap with another objects boundingbox. (as long as each object has such an BB area...)
If u the use an inside or centre point on those objects, it is mostly possible to still get a clean sorting order using the expression i posted.
Another way would be to use distance of objects to oneother and use e Thiessenpolygon creator and extract the inside or center point form those.
+15- Contributor
Addendum.
Boundingbox could also be used to find the point on a object (for instance ) upper left corner.
You first create BB's using the BB accumulator grouped by ID.
Extract bounds using a boundsextractor.
Create a line from left upper to right lower corner (or visa versa).
Coerce original bounds to line.
Intersector (or linonlineoverlayer) to acquire intersection points.
Choose upper left (or anyother you prefer).
Latter you can do by using a statistics calculator grpuped by object ID find max_Y and min_x (for upper left) then a tester to x= min_x AND y=max_y.
This way you don't need to gamble on say 1st (or last) vertex when you dont know where they will be.
Again this will be dependand on geometric shape of objects.
If you want to truly rule this geometric dependancy out, you have to segement to bounds of an object to great resolution and do a statistics on all the vertices. (then i would advice a tiling/paralel processing strategy because it will need a lot of processing...)
gio wrote:
@gsteinmon
To sort objects from left to right you cabn use goniometry.
I fiddled for couple of hours (3-ish morel like) and found this one.
1. Create centerpoint for objects (or inside point).
2. Extract vertex.
3. And then create following value using the expression evaluator:
@Value(_y)-@atan(@Value(_x)/@Value(_y))*10000
Sort by this value and then use a count to create a attribute for order (else you be looking at largish numbers, counter makes it bit tidier)
The value 1000 is to garantee sufficient resolution, so each object has a unique value. (if u have zillions of objects you might need to increase this value)
The expression is for left to right, top down.
With little fidling you can reverse the order or make it right to left (in case you're like Asian...)
I made this years ago so people can find objects on maps easier.
haves fun,
+44- Safer
I just revisited this question and thought I would add that there is now a SpatialSorter transformer in the FME Hub. It doesn't sort east-west, but it uses a repeating shape to sort data so that features are sorted in order of closeness.
+44- Safer
chiron wrote:
I stumbled on this answer by accident and found that it reduced the execution time of my PointOnRasterValueExtractor over a large number of points (with a very large raster on an external drive) by more than half!!!
mark2atsafe wrote:
Excellent! I'm not sure I know why, but I'm glad to hear it was helpful.
I struggled with this problem a lot during the last few days. The investigation revealed that Active Time to the external Hard Drive was at a constant 100% and a single CPU core was not at max. I suspected that the PointOnRasterValueExtractor uses some form of offset reading, as within the time to startup it could obviously not read the full 760GB of my source image into memory or some form of cache. Hence my hunch that if I could sort my source points according to offset, image access will be faster. This hunch paid off significantly. I immediately noticed that Active Time dropped to about 70%, and the Average Response Time dropped from about 35ms to about 5ms. Furthermore, CPU execution climbed to the max for single core execution which led me to believe that my workspace is now processer limited and no longer disk limited. I suspect that the head of the Hard Drive can now move in smaller increments instead of pure random access, hence the drop in Response Time.
Anyway, I also suspect that If I were able to move the full image to a Solid State Drive that this advantage will disappear. But it might be useful to someone in a similar situation to me. On a larger random sample of 500 000 points, execution time dropped by 70%!
PS By using this method after a PointCloudCoercer instead of a PointCloudOnRasterComponentSetter on my original point cloud set, caused another order of magnitude time gain.
Reply
Helpful Members This Week
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.