Question

How to stop a DEM's no data values from being filled in?


What transformer would I need, to stop the RasterDEMGenerator from filling in areas that should be void, or what other transformer(s) should I use?

 

I am using a CSV with XYZ values as my input and in some cases there are voids/no data/holes in the data see pics. How do I stop FME from interpreting data that isn't there? Output is a Geotiff.


12 replies

Userlevel 2
Badge +19

Perhaps use RasterBandNodataRemover transformer. See the example in the documentation

Userlevel 6
Badge +33

Not sure if this is the best way, but you can use the NumericRasterizer to create a raster with null values on a parallel stream, then combine the two using the RasterExpressionEvaluator to set the cell value to null when the second raster cell value is null.

Perhaps use RasterBandNodataRemover transformer. See the example in the documentation

I had a look at that transformer, unless I'm missing something. The RasterDEMGenerator has already filled in the areas of no data with calculated values, so from this point onwards it wouldn't know what is meant to be no-data and what is actual data. Hope that makes sense.

Not sure if this is the best way, but you can use the NumericRasterizer to create a raster with null values on a parallel stream, then combine the two using the RasterExpressionEvaluator to set the cell value to null when the second raster cell value is null.

Nice Idea, I see where you are going with this, but I think the same problem as above applies?

Could I create a polygon from the VertexCreator's data to clip or erase the raster area of no-data?

Userlevel 6
Badge +33

Nice Idea, I see where you are going with this, but I think the same problem as above applies?

Attached a workspace demonstrating how I would do this.

2022-06-13_10h49_05

Nice Idea, I see where you are going with this, but I think the same problem as above applies?

Thanks for that, interestingly I get this error "Raster A has '96' rows and '78' columns, whereas raster B has '97' rows and '79' columns. Paired rasters must have the same dimensions"

Nice Idea, I see where you are going with this, but I think the same problem as above applies?

Here is my CSV data

Userlevel 6
Badge +33

Nice Idea, I see where you are going with this, but I think the same problem as above applies?

This is caused by different extents. I modified your workspace and I think it does what you need. I also shifted the inputpoints 0.5 cell for the numeric input as the results were shifted. I'm not sure why tho, maybe someone can chime in to correct me if I'm wrong.

Nice Idea, I see where you are going with this, but I think the same problem as above applies?

Awesome, thanks for that. Ill have a look through other CSV's I have, to see if the shift creates any inaccuracies. Many thanks for your help.

Userlevel 6
Badge +33

Nice Idea, I see where you are going with this, but I think the same problem as above applies?

@Andrew Needham​ I got better results when I convert the vertices to pointcloud using a PointCloudCombiner. Now the cells align with the data.

 

Also, reading the file as pointcloud (format Point Cloud XYZ) is probably much quicker than reading the file as csv and then create vertices from it.

Nice Idea, I see where you are going with this, but I think the same problem as above applies?

Yeah, the point cloud works much better. I was finding irregularities with the earlier version(raster cells where there was no vertex's/points). I did look into another direction which square buffered the points to be used as a clip, this also works. It was good to see how some of the Trasformers worked like the RasterExperessionEvaluator, I would never have gotten that expression to work, cheers.

 

One issue I have is when publishing to FME Server the CSV inputs X,Y,Z values are not dynamic, as in they are not returning the first row as headers, as they do in workbench. I know Server doesn't work the same as desktop, but there should be a way to add any CSV files and either select a header name or position col1,col2 etc for the values required.

 

I just need to investigate further :)

Reply