Question

FME ERROR: TIFF writer: The resulting file is too big

  • 25 November 2013
  • 7 replies
  • 33 views

Badge
Hi,

 

 

I am trying to combine 2077 Raster images into one big one. I chose TIFF as the output file format which threw this error. 

 

 

(SYSTEM: Desktop FME 2013 on Win7 4gb Ram)

 

 

"TIFF writer: The resulting file is too big (approximately 207.87 GB).  This format cannot support a file size over 4.00 GB.  Consider tiling the raster to reduce the output file size"

 

 

Nice error message, but what is my alternative? Should I use the BigTIFF format?

 

 

Thanks,

 

 

Rob

7 replies

Userlevel 4
Hi,

 

 

yes indeed, the TIFF standard imposes a limit of 4GB. BigTIFF might be an option but do check for compatibility on the client side, not all systems support it.

 

 

As the message states, also consider using the Tiler to split your output over several, smaller TIFFs. This might also be a good idea from a performance standpoint.

 

 

Another option might be the JPEG2000, which may give significantly smaller files for certain images, particularly "monotone" orthophotos. Just be aware that not that many clients support it and that the compression is lossy (but configurable).

 

 

David
Userlevel 4
Badge +13
Hi,

 

 

Another format option in ECW, but since the compression also eats up lots of memory resources, consider David's recommendations.

 

 

Itay
Badge
Im still having problems creating one image.

 

 

Error message is not too clear.

 

 

"TIFF writer: TIFFAppendToStrip:Write error at scanline 24256

TIFF writer: TIFFAppendToStrip:Write error at scanline 24272

TIFF writer: An error occured while writing a dirty block

TIFF writer: TIFFAppendToStrip:Write error at scanline 24304

TIFF writer: TIFFAppendToStrip:Write error at scanline 24320

... Last line repeated 2 times ...

TIFF writer: Output file creation failed

A fatal error has occurred. Check the logfile above for details

... Last line repeated 2 times ...

RasterMosaicker(RasterMosaicFactory): Failed to output the feature with the mosaicked raster

A fatal error has occurred. Check the logfile above for details

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Feature output statistics for `TIFF' writer using keyword `TIFF_2':

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Features Written

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

dop2013 1

==============================================================================

Total Features Written 1

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

A fatal error has occurred. Check the logfile above for details

FME Session Duration: 9 minutes 35.9 seconds. (CPU: 320.4s user, 25.6s system)

END - ProcessID: 7060, peak process memory usage: 474236 kB, current process memory usage: 72996 kB

 

A fatal error has occurred. Check the logfile above for details

Program Terminating

 

Translation FAILED."

 

 

I have checked the box "Create Tiled Files" and also "enable BigTIFF" as well as lzw compression.

 

 

These are 3 band Orthophotos - 405 tiles (not 2077 like I wrote earlier - that was another problem with 1:5000 Topographic Maps). Each Tile is a tif on about 307mb 20cm resolution.  

 

 

Should I be using any other transformers here? At the moment I´ve just got a reader -> rasterMosaiker->writer.

 

 

 

Thanks for any help,

 

 

RB

 

 

 

 
Userlevel 4
Badge +13
a sorter would be advisable, to make sure that the order of the tiles is correct.
Badge
Have tried again this time with a sorter, but I think FME has a problem with the total file size.

 

 

Oh well....have to go bvack to ArcGIS for this one.

 

 

Thanks for your help,

 

Rob

 

 

Userlevel 4
Badge +13
Hi,

 

 

I doubt ArcGIS will be any better, it can be that your memory resources are not allowing this operation to complete, search for temp on the FMEpedia.

 

 

In the past I have created a mosaic made out of ca. 2000 tiles (each 50MB) with FME, it was done on a 64 bit machine with 16G memory, also creating sub mosaics and clearing the memory (RasterCheckpointer) was needed.

 

 

See http://fmepedia.safe.com/articles/How_To/Tips-for-mosaicking-large-raster-datasets

 

 for more info.

 

 

Itay
what tiff writer software are you using . if you merge tiff files, you will certainly get a large size. however you can compress

Reply