Solved

Count distinct/unique component values of a pointcloud

james_c_452
Rank
Badge +4

Hi,

 

I have a pointcloud where I want to count the unique number of values in a component. In this case I want to find the number of unique GPS time values in a point cloud (The point cloud is made from individual epochs of a lidar scanner, but there are time gaps so I cant use the max/min values)

 

Basically is there something like the hub transformer AttributeValueCounter (which doesn't work on components). I do not know how to use python so I can't work out how to edit their method to work on components rather than attributes

 

I can kind of do it by using the PointCloudSplitter and counting the files, but this is really slow

 

Thanks

icon

Best answer by evieatsafe 7 March 2023, 01:26

View original

7 replies

evieatsafe
Rank
Userlevel 1
Badge +15

Hi @james_c_452​, I was actually going to suggest using the PointCloudSplitter which can be slow depending on how large your point cloud is. Another method of doing this if you know the component values you want to filter for is to use the PointCloudFilter which uses conditional statements on component values. This may take longer to set up depending on how many filters you want to use, but might be faster to run. Hope this helps!

Happy to help, Evie
james_c_452
Rank
Badge +4

Hi @james_c_452​, I was actually going to suggest using the PointCloudSplitter which can be slow depending on how large your point cloud is. Another method of doing this if you know the component values you want to filter for is to use the PointCloudFilter which uses conditional statements on component values. This may take longer to set up depending on how many filters you want to use, but might be faster to run. Hope this helps!

Hi @Evie Lapalme​ ,

My point clouds are quite large. Billions of points. So I'd like to avoid the PointCloudSplitter.

How would I use the PointCloudFilter? I haven't been able to think of anything. Basically I want to count the unique time stamps (I've rounded them to the nearest second in a new component first). I was thinking maybe I could filter the first point for each time by comparing a point's component to the previous point, but I'm not sure how you would do this. I don't think there is a way to compare components like this, like you can with attributes

Thanks

evieatsafe
Rank
Userlevel 1
Badge +15

Hi @Evie Lapalme​ ,

My point clouds are quite large. Billions of points. So I'd like to avoid the PointCloudSplitter.

How would I use the PointCloudFilter? I haven't been able to think of anything. Basically I want to count the unique time stamps (I've rounded them to the nearest second in a new component first). I was thinking maybe I could filter the first point for each time by comparing a point's component to the previous point, but I'm not sure how you would do this. I don't think there is a way to compare components like this, like you can with attributes

Thanks

I think it would depend on how many unique time stamps you want to collect, if you want them all to be individual then perhaps the PointCloudSplitter is best. However, if you want to set ranges of timestamps then the PointCloudFilter might be better. It all depends on what you want to do next and what your expected outcome is. In the PointCloudFilter you can set expressions based on component values similar to the PointCloudExpressionEvaluator transformer.

 

For an example on how to set expressions see this article: How to Clean Up Classification Errors in Point Clouds. That article also gives you some other methods you could use to extract different parts of your point cloud, e.g. using the PointCloudToPointCoercer transformer. Some of those methods are not suited to large datasets but are also options. If you attempt those methods I would tile your point cloud dataset with a Tiler transformer first. Hope this helps!

Happy to help, Evie
james_c_452
Rank
Badge +4

Hi @Evie Lapalme​ ,

My point clouds are quite large. Billions of points. So I'd like to avoid the PointCloudSplitter.

How would I use the PointCloudFilter? I haven't been able to think of anything. Basically I want to count the unique time stamps (I've rounded them to the nearest second in a new component first). I was thinking maybe I could filter the first point for each time by comparing a point's component to the previous point, but I'm not sure how you would do this. I don't think there is a way to compare components like this, like you can with attributes

Thanks

Hi @Evie Lapalme​ ,

Thanks for the reply. I hope this response makes sense

 

What I have is a trajectory of a lidar system. So a point cloud file at time stamp increments of where an actual point cloud was stored from. It records at least once a second, up to 200 times, depending on my supplier. I want to calculate the actual frequency that it collects data so I can thin it as appropriate to a target frequency.

 

So what I have done is compute a component with each time stamp rounded to the nearest second. Sometimes the trajectory is given to me as a fully sequential file (so I can just use the max/min values of the time to compute how many seconds have recorded data). Sometimes it is given to me not as a continuous file, but with time gaps inside. I want to count the unique seconds where there is data to account for these gaps.

 

There are thousands of unique seconds which will take the PointCloudSplitter a while to split the file. It would be a bit quicker if I could, for example, split into time ranges of 5 seconds. Though I don't not know how to set this up without having to manually set hundreds of time windows each time.

 

If I use the tiler first I could parallel process by a tile ID, though this would get multiple ranges because of parallel rails. It would be better if I could split into time windows, or remove a point from a cloud if it matches the previous point (which would leave only one point for each time stamp). Or if I could perhaps use the PointCloudThinner to thin by 1 point per second. I can Coerce the points, but I am trying to thin the data to a target frequency first, so I don't need to coerce hundreds of thousands of points

evieatsafe
Rank
Userlevel 1
Badge +15

Hi @Evie Lapalme​ ,

My point clouds are quite large. Billions of points. So I'd like to avoid the PointCloudSplitter.

How would I use the PointCloudFilter? I haven't been able to think of anything. Basically I want to count the unique time stamps (I've rounded them to the nearest second in a new component first). I was thinking maybe I could filter the first point for each time by comparing a point's component to the previous point, but I'm not sure how you would do this. I don't think there is a way to compare components like this, like you can with attributes

Thanks

Hi again @james_c_452​, this does make sense thanks for elaborating :) I talked to a colleague and we agree that we should get away from the PointCloudSplitter and PointCoercer transformers if we can because of performance.

 

I think we can solve this with a bit of experimentation and a combination of transformers such as: PointCloudPropertyExtractor, PointCloudExpressionEvaluator, PointCloudSorter, PointCloudStatisticsCalculator, and PointCloudMerger. You could also involve the conversion from point cloud to raster as they are treated similarly. If you could provide us with some sample data we might be able to experiment with you on this, hard to tell what will work specifically with your data, what expressions to use, etc.

 

The PointCloudMerger is one of the more powerful transformers for simplifying workspaces as it will merge everything based on what its matching and only output points where the match key is unique. So we might be able to match on time and just keep the first point per time for example.

Happy to help, Evie
james_c_452
Rank
Badge +4

Hi @Evie Lapalme​ ,

My point clouds are quite large. Billions of points. So I'd like to avoid the PointCloudSplitter.

How would I use the PointCloudFilter? I haven't been able to think of anything. Basically I want to count the unique time stamps (I've rounded them to the nearest second in a new component first). I was thinking maybe I could filter the first point for each time by comparing a point's component to the previous point, but I'm not sure how you would do this. I don't think there is a way to compare components like this, like you can with attributes

Thanks

Hi @Evie Lapalme​ ,

I've attached a txt file with a few gaps in it. What I do for here is read it in as a point cloud (which is much quicker than the individual points), and then round the time to a new component (using the PointCloudExpressingEvaluator, Uint64). I have tried the PointCloudMerger/sorter before but I couldn't get it to work. I'm probably doing it wrong. In the case where the file is continuous I get the range from the PointCloudStatisticsCalculator to get the unique number of seconds (which I can then divided by the total number of points to get the Hertz). I'll have another play when I next get the chance

1 Attachment
james_c_452
Rank
Badge +4

Hi @Evie Lapalme​ ,

My point clouds are quite large. Billions of points. So I'd like to avoid the PointCloudSplitter.

How would I use the PointCloudFilter? I haven't been able to think of anything. Basically I want to count the unique time stamps (I've rounded them to the nearest second in a new component first). I was thinking maybe I could filter the first point for each time by comparing a point's component to the previous point, but I'm not sure how you would do this. I don't think there is a way to compare components like this, like you can with attributes

Thanks

Hi @Evie Lapalme​ ,

I have had another go at using the PointCloudMerger. Looks like I was using the wrong output port. I was using the Merged port, which seems to include everything. The Referenced port appears to have only one output per (rounded) time stamp. So as you said, and what I couldn't get previously to work. I combined the PointCloudMerger with a PointCloudExtractor before and after the merger to get the number of points in each case (and thus the Hertz/Frequency I was after)

Thanks for your help!

Reply


Community Stats

28,694
Posts
109,434
Replies
36,545
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.