Skip to main content
Question

Raster merge


lazarlubomir
Contributor
Forum|alt.badge.img+7

Hello,

I have more than 300 TIFF files and I need to merge them into 1 TIFF/ECW file... Is there any way how to merge them please?

Thank You so much

Lubo

16 replies

david_r
Evangelist
  • July 21, 2016

lazarlubomir
Contributor
Forum|alt.badge.img+7
  • Author
  • Contributor
  • July 22, 2016
david_r wrote:

@david_r Thank You for tip! But what if translation failed because of not the same bands number? Can I fix it this problem in FME? Check please pic below from logfile:

Log:

2016-07-22 08:45:42| 29.4| 0.2|ERROR |RasterMosaicker(RasterMosaicFactory): All input rasters must have the same number of selected bands. The raster '80' has '1' selected bands, but the mosaicked raster '0' has '3' selected bands

2016-07-22 08:45:43| 30.0| 0.6|ERROR |A fatal error has occurred. Check the logfile above for details

2016-07-22 08:45:43| 30.0| 0.0|ERROR |


david_r
Evangelist
  • July 22, 2016

Try adding a RasterSelector before the RasterMosaicker.

If that isn't enough, then your rasters differ too much to mosaick as-is, and you'll need to do some pre processing.

Relevant transformers could be the RasterPropertyExtractor, check exit attribute _num_bands in particular, it should be the same for all your input rasters. If necessary, look into the RasterBandAdder, RasterBandRemover, etc.


lazarlubomir
Contributor
Forum|alt.badge.img+7
  • Author
  • Contributor
  • July 22, 2016
david_r wrote:

Try adding a RasterSelector before the RasterMosaicker.

If that isn't enough, then your rasters differ too much to mosaick as-is, and you'll need to do some pre processing.

Relevant transformers could be the RasterPropertyExtractor, check exit attribute _num_bands in particular, it should be the same for all your input rasters. If necessary, look into the RasterBandAdder, RasterBandRemover, etc.

@david_r Great idea, thank You, it works to define, which raster has one or three bands. Anyway, my process was stopped because of lack of space on disk. I wanted to merge via rastermosaicker approx. 250 geoTIFF rasters into one ECW file. Each raster is 90 MB large and I have 60 GB free space on my disc. Approx. after creating of 10 % of ECW file process is stopped because of no space on disc. I checked it, the FFS file has more than 50 GB! Is there any possibility to stop exporting of FFS or any possibility to compress this file?

Thank You so much!


lazarlubomir
Contributor
Forum|alt.badge.img+7
  • Author
  • Contributor
  • July 22, 2016
david_r wrote:

Try adding a RasterSelector before the RasterMosaicker.

If that isn't enough, then your rasters differ too much to mosaick as-is, and you'll need to do some pre processing.

Relevant transformers could be the RasterPropertyExtractor, check exit attribute _num_bands in particular, it should be the same for all your input rasters. If necessary, look into the RasterBandAdder, RasterBandRemover, etc.

@david_r And one more question - is in FME possibility to create three band raster (RGB) from single band raster (UINT8) please?


david_r
Evangelist
  • July 22, 2016
lazarlubomir wrote:

@david_r Great idea, thank You, it works to define, which raster has one or three bands. Anyway, my process was stopped because of lack of space on disk. I wanted to merge via rastermosaicker approx. 250 geoTIFF rasters into one ECW file. Each raster is 90 MB large and I have 60 GB free space on my disc. Approx. after creating of 10 % of ECW file process is stopped because of no space on disc. I checked it, the FFS file has more than 50 GB! Is there any possibility to stop exporting of FFS or any possibility to compress this file?

Thank You so much!

GeoTIFFs are usually compressed with a very efficient compression algorithm, but when e.g. FME wants to mosaick the files it needs to uncompress them and work with the raw data, which can lead to huge temporary files. This goes even if you intend to write it back into another format that supports compression, such as ECW.

Also, consider not only getting more disk space, but also using the 64-bit version of FME for this task.

But on another note: Even if you were to manage to create this humongous ECW raster, are you sure that your target application will handle it well?


david_r
Evangelist
  • July 22, 2016
lazarlubomir wrote:

@david_r And one more question - is in FME possibility to create three band raster (RGB) from single band raster (UINT8) please?

Try the RasterBandAdder.


lazarlubomir
Contributor
Forum|alt.badge.img+7
  • Author
  • Contributor
  • July 22, 2016
david_r wrote:

@david_r Yes, it works, but I need the new three band raster (RGB) has the same values like single band raster (source). If I add new three band - red, green, blue nad then remove UINT8 band (to be able to connect raster to rastermosaicker), then raster is whole black. All I need is to create three-band raster from single-band raster with the same visualization. Is there any possibility please?


lazarlubomir
Contributor
Forum|alt.badge.img+7
  • Author
  • Contributor
  • July 22, 2016
david_r wrote:

GeoTIFFs are usually compressed with a very efficient compression algorithm, but when e.g. FME wants to mosaick the files it needs to uncompress them and work with the raw data, which can lead to huge temporary files. This goes even if you intend to write it back into another format that supports compression, such as ECW.

Also, consider not only getting more disk space, but also using the 64-bit version of FME for this task.

But on another note: Even if you were to manage to create this humongous ECW raster, are you sure that your target application will handle it well?

You´re right. In this case I will probably use this way: https://knowledge.safe.com/articles/176/fme-temp-environment-variable.html... I use 64-bit version of FME, but will connect external disk to get more space. Target application one time worked with 2TB raster, so it should do it right :-)


david_r
Evangelist
  • July 22, 2016
lazarlubomir wrote:

@david_r Yes, it works, but I need the new three band raster (RGB) has the same values like single band raster (source). If I add new three band - red, green, blue nad then remove UINT8 band (to be able to connect raster to rastermosaicker), then raster is whole black. All I need is to create three-band raster from single-band raster with the same visualization. Is there any possibility please?

If you add three empty bands and then delete the fourth that contains your actual cell values, then yes, the raster will be pretty dull to look at :-)

I've posted two solutions below (necessary because of forum limitations).


david_r
Evangelist
  • July 22, 2016

There are (at least) two ways to convert a single-band raster into a multi-band raster.

If the existing band represents a known color, you force (coerce) the existing band to the complementary color and add the two remaining ones. This example takes a grayscale band and converts it to a cyan 3-band raster:

The other solution is to make three copies of your grayscale band, force each to either red/green/blue and then combine them into a single 3-band raster:

This will preserve the grayscale (unless there are palettes involved, in which case you can just use the RasterPaletteResolver to convert directly to RGB).


lazarlubomir
Contributor
Forum|alt.badge.img+7
  • Author
  • Contributor
  • July 22, 2016
david_r wrote:

There are (at least) two ways to convert a single-band raster into a multi-band raster.

If the existing band represents a known color, you force (coerce) the existing band to the complementary color and add the two remaining ones. This example takes a grayscale band and converts it to a cyan 3-band raster:

The other solution is to make three copies of your grayscale band, force each to either red/green/blue and then combine them into a single 3-band raster:

This will preserve the grayscale (unless there are palettes involved, in which case you can just use the RasterPaletteResolver to convert directly to RGB).

@david_rThank You for tips! I tried to use RasterBandInterpretationCoercer this morning, but log shows me, that I cant use this transformer for raster with transformer... Take a look below please:

@ReinterpretRaster: In BAND mode, if selected bands contain palettes then the requested interpretation may only be one of 'UINT8, UINT16, UINT32'

2016-07-22 15:16:30| 0.8| 0.0|ERROR |RasterBandInterpretationCoercer: A fatal error has occurred. Check the logfile above for details


david_r
Evangelist
  • July 22, 2016
lazarlubomir wrote:

You´re right. In this case I will probably use this way: https://knowledge.safe.com/articles/176/fme-temp-environment-variable.html... I use 64-bit version of FME, but will connect external disk to get more space. Target application one time worked with 2TB raster, so it should do it right :-)

Regarding an external disk, just be aware that the connection speed for this disk may have a significant impact on performances. I would normally discourage using network drives and external disks connected by USB 2.


david_r
Evangelist
  • July 22, 2016
lazarlubomir wrote:

@david_rThank You for tips! I tried to use RasterBandInterpretationCoercer this morning, but log shows me, that I cant use this transformer for raster with transformer... Take a look below please:

@ReinterpretRaster: In BAND mode, if selected bands contain palettes then the requested interpretation may only be one of 'UINT8, UINT16, UINT32'

2016-07-22 15:16:30| 0.8| 0.0|ERROR |RasterBandInterpretationCoercer: A fatal error has occurred. Check the logfile above for details

Sounds like you may have palettes in your rasters. Try skipping the coercers/combiner and just use a RasterPaletteResolver, as I mentioned above.


lazarlubomir
Contributor
Forum|alt.badge.img+7
  • Author
  • Contributor
  • July 22, 2016
david_r wrote:

Sounds like you may have palettes in your rasters. Try skipping the coercers/combiner and just use a RasterPaletteResolver, as I mentioned above.

Yes, the problematic rasters contain palletes... And how can I see right now, RasterpaletteResolver is solution :-) Thank You so much for help!


Forum|alt.badge.img
david_r wrote:

There are (at least) two ways to convert a single-band raster into a multi-band raster.

If the existing band represents a known color, you force (coerce) the existing band to the complementary color and add the two remaining ones. This example takes a grayscale band and converts it to a cyan 3-band raster:

The other solution is to make three copies of your grayscale band, force each to either red/green/blue and then combine them into a single 3-band raster:

This will preserve the grayscale (unless there are palettes involved, in which case you can just use the RasterPaletteResolver to convert directly to RGB).

Thanks for the RasterPaletteResolver hint! This made my 3D PDF workflow much faster, instead of resampling the raster before putting it into the AppearanceSetter, I now could just run it maybe five to ten times faster.

 

For ECW output (8-bit needed) I used the RasterBandInterpretationCoercer, linked directly to the output port. Worked fine!

Thanks! Have a nice weekend!


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings