Skip to main content
  • EN
Solved

Fill the gaps raster image

How can I use FME parameters to fill gaps or holes in a raster image? Please explain best methods. 

 

15 replies

virtualcitymatt
Rank
Userlevel 5
Badge +32

So there are a few options depending on what you want to achieve. 

1. Fill the background with another georeferenced image

     - You can stitch this raster and another raster together with the RasterMosaicker. This requires that both images are georeferenced

2. In case the background it transparent:
    - If your background is transparent in your application it means that it has an ALPHA band. You can use the RasterInterpretationCoercer to go from an RGBA raster to an RGB raster. This will most likely set the hole to black.

3. Set a specific color:
    - If you want the white to be a specific color you can use the method used here:
 



 

redgeographics
Rank
Userlevel 6
Badge +35

The only way I can think of with FME is to take the original raster, downsample it (RasterResampler) so that one pixel is larger than that gap and then mosaick it (RasterMosaicker) with the original, making sure you keep the original on top and turn the white pixels transparent (if they aren’t already transparent)

The result is probably an image where the gap is filled with a solid color and you’ll most likely have artifacts along the edge of the gap, I don’t think it’s going to look good to be honest.

mahapandian
Rank
Badge +2

Hi @redgeographics, @virtualcitymatt Thank you for your clarification. It sounds like we are working with a large number of drone image datasets containing gaps or holes that I want to fill automatically using average pixel value from neighbour pixels using FME Parameters.

  1. Test Case I – Raster Hole filling for waterbodies based on waterbody boundaries given by user
  2. Test Case II – Finding, Geotagging, encircling areas with missing pixels/mosaic transparency.
dmitribagh
Rank
Userlevel 4
Badge +14

Hi @mahapandian,

 

for finding areas with missing pixels, you can use RasterExpressionEvaluator with the following expression:

if(A[0]==0&&A[1]==0&&A[2]==0,1,0)
It says, replace all pixels with 0 values in RGB bands with ones, and set the rest to 0. Then, set Nodata to 0 (RasterBandNoDataSetter) followed by RasterExtentsCoercer (‘Data Extents”). This will give you vector representation of all holes as a single aggregate. Then - Deaggregator and CenterPointReplacer - this will give you points. You can buffer them to get circles (or rather their polygon approximations).

For filling bodies of water, you probably can simply find a raster showing water only, clip your main rasters with water boundaries, and then mosaic with the water-only raster sitting below your actual rasters.

Dmitri

mahapandian
Rank
Badge +2

@dmitribagh Thank you for your valuable suggestion. I will try what you suggested. However, I cannot use raw images for the mosaic because a drone image isn't a single tile covering the area. it consists of multiple images stitched together to form a single tile (orthomosaic), often containing thousands of images in one tile.

My requirement is to automatically identify null values, holes, or gaps within the drone RGB data and then fill those gaps using neighboring pixels or any other methods using FME transformers. 

dmitribagh
Rank
Userlevel 4
Badge +14

Hi @mahapandian,

Here is what I tried. RasterConvolver can fill small holes by setting the average value of the surrounding pixels on the center pixel, however, this method slightly blurs the whole image. So, I identified all holes as I described above, made a small buffer around them (buffer size = image resolution), clipped the raster with these small buffers and got all the holes with a frame of data pixels as separate small rasters. Then, I calculated statistics (RasterStatisticsCalculator) - mean value for each band, and set these calculated values to all pixels of these small rasters (RasterExpressionEvaluator). Then I mosaicked everything together, and here are a few examples how it all looks (left - before, right - after):

As you can see, if a hole is surrounded by pixels with similar RGB values, it works quite well. When there is a big difference, the output does not look that well - check the last example. 

I hope this helps. If you have any question how it works or need more brainstorming, feel free to let me know.

Dmitri

dmitribagh
Rank
Userlevel 4
Badge +14

Hi @mahapandian,

 

I also tried a Python OpenCV approach, and it seems to work quite well, have a look at this animated GIF (image is too big to insert it here, about 10Mb):

https://safe-scenarios.s3.us-west-2.amazonaws.com/images/filledHoles.gif

 

If you would like to explore this possibility, please let me know - I have this workflow in a messy state, but can clean it up for publishing.

 

Dmitri

mahapandian
Rank
Badge +2

Hi @mahapandian,

Here is what I tried. RasterConvolver can fill small holes by setting the average value of the surrounding pixels on the center pixel, however, this method slightly blurs the whole image. So, I identified all holes as I described above, made a small buffer around them (buffer size = image resolution), clipped the raster with these small buffers and got all the holes with a frame of data pixels as separate small rasters. Then, I calculated statistics (RasterStatisticsCalculator) - mean value for each band, and set these calculated values to all pixels of these small rasters (RasterExpressionEvaluator). Then I mosaicked everything together, and here are a few examples how it all looks (left - before, right - after):

As you can see, if a hole is surrounded by pixels with similar RGB values, it works quite well. When there is a big difference, the output does not look that well - check the last example. 

I hope this helps. If you have any question how it works or need more brainstorming, feel free to let me know.

Dmitri

Hi @dmitribagh Thank you very much for your detailed clarification of my requirements. I'll give it a shot. 

mahapandian
Rank
Badge +2

Hi @mahapandian,

 

I also tried a Python OpenCV approach, and it seems to work quite well, have a look at this animated GIF (image is too big to insert it here, about 10Mb):

https://safe-scenarios.s3.us-west-2.amazonaws.com/images/filledHoles.gif

 

If you would like to explore this possibility, please let me know - I have this workflow in a messy state, but can clean it up for publishing.

 

Dmitri

@dmitribagh Absolutely, I'd be happy to walk you through the workflow. Would you be able to share it with me?

dmitribagh
Rank
Userlevel 4
Badge +14

Hi @mahapandian 

Here is the template with my experiment.

https://www.dropbox.com/scl/fi/f2c26xkcuie5ttygumret/FillingHoles.fmwt?rlkey=7yihakj0issfgn7h6nf4icuu6&dl=0

You will need to install OpenCV Python library. See the command line for this in the workspace.

Here is the map that shows the results of the workspace run. Use 1 and 2 keys to switch between before/after images. Zoom in to any marker or use navigation buttons on top to go through all of them (might be boring 😁 - it’s so easy to generate as many holes as we want):

https://safe-scenarios.s3.us-west-2.amazonaws.com/Leaflet/HoleInpaint/inpaintReview.html
I hope this helps with smaller gaps or even bigger gaps on areas with not very many diverse details.

I will make a LinkedIn post showing this integration for other users to try this functionality.

Dmitri

 

mahapandian
Rank
Badge +2

Hi @mahapandian 

Here is the template with my experiment.

https://www.dropbox.com/scl/fi/f2c26xkcuie5ttygumret/FillingHoles.fmwt?rlkey=7yihakj0issfgn7h6nf4icuu6&dl=0

You will need to install OpenCV Python library. See the command line for this in the workspace.

Here is the map that shows the results of the workspace run. Use 1 and 2 keys to switch between before/after images. Zoom in to any marker or use navigation buttons on top to go through all of them (might be boring 😁 - it’s so easy to generate as many holes as we want):

https://safe-scenarios.s3.us-west-2.amazonaws.com/Leaflet/HoleInpaint/inpaintReview.html
I hope this helps with smaller gaps or even bigger gaps on areas with not very many diverse details.

I will make a LinkedIn post showing this integration for other users to try this functionality.

Dmitri

 

Hi @dmitribagh Thank you so much. I will try as you suggested. 

mahapandian
Rank
Badge +2

Hi @mahapandian 

Here is the template with my experiment.

https://www.dropbox.com/scl/fi/f2c26xkcuie5ttygumret/FillingHoles.fmwt?rlkey=7yihakj0issfgn7h6nf4icuu6&dl=0

You will need to install OpenCV Python library. See the command line for this in the workspace.

Here is the map that shows the results of the workspace run. Use 1 and 2 keys to switch between before/after images. Zoom in to any marker or use navigation buttons on top to go through all of them (might be boring 😁 - it’s so easy to generate as many holes as we want):

https://safe-scenarios.s3.us-west-2.amazonaws.com/Leaflet/HoleInpaint/inpaintReview.html
I hope this helps with smaller gaps or even bigger gaps on areas with not very many diverse details.

I will make a LinkedIn post showing this integration for other users to try this functionality.

Dmitri

 

Hi @dmitribagh Thank you very much for your support. I've tried using your workflow to fill in the gaps, but unfortunately, I haven't been able to resolve the issues.

 

mahapandian
Rank
Badge +2

Hi @dmitribagh Could you please assist me in resolving this issue? I have imported all the libraries you previously mentioned, but I am unable to import 'fme' and 'fmeobjects' (as highlighted in the yellow marked image).

Kindly provide a step-by-step explanation, focusing on the PythonCaller transformer.

Thanks for your support. 

 

dmitribagh
Rank
Userlevel 4
Badge +14

Hi @mahapandian,

 

I am not a Python expert by any standard, but fme and fmeobjects should load just by the fact that you run PythonCaller, unless you switched from FME Python to some other version, and it feels like this might be the case - your Python error points at some location where FME usually does not install libraries.

 

Usually, if you install a Python library through FME, it goes to (on Windows):

%UserProfile%\Documents\FME\Plugins\Python

and on Mac, I found opencv here (for FME. 2023.2):

/Users/<USER>/Library/Application Support/FME/Plugins/Python/python311

Could you verify that you are using FME Python?

 

Dmitri

dmitribagh
Rank
Userlevel 4
Badge +14

Hi @mahapandian,

 

I was able to reproduce a similar error:
error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'

I am currently testing different raster interpretations, and found out that, for example, RGB24, RGBA32, RGB48, UINT8, INT16, RED8, etc. work equally well, but Real64 or UINT32 interpretations make the script fail and cause to this error. I also tried a custom interpretation - RGBAUInt40, and it also led to the error.

I wonder what interpretation your raster has.

You can adjust the interpretation with RasterInterpretationCoercer - choose one of the supported ones. Note that the output will be an RGB24, you can use RasterInterpretationCoercer to restore the original interpretation.

Or, if you have a non-standard band configuration, you can split the raster, keep the RGB bands for processing and then combine the bands back together. (RasterBandSeparator and RasterBandCombiner).

 

Dmitri

Reply


Community Stats

29,620
Posts
112,572
Replies
37,269
Members
Terms & ConditionsCookie settings