Skip to main content

I'm finally posting here after much experimenting with various raster transformers and not really getting anywhere. There are too many permutations of things I've tried to list here, so I'll just outline my requirements.

I have a set of scanned images of map sheets comprised of JPEGs which are 3 band RED8, GREEN8, BLUE8, no palette and TIFFs which are UINT8, 1 palette. I also have a CSV file with georeferencing and clipping data. I can successfully apply the georeferencing data and clip the images.

I want to output the results in 2 ways. Firstly, a set of full colour (3 band RED8, GREEN8, BLUE8 in FME parlance?) TIFFs with TFW world files. Secondly, as a mosaicked image in ESRI format - File Geodatabase Raster and/or ArcSDE raster.

When I load the output TIFFs that came from JPEG source images into FGDB rasters and display them in ArcGIS, dark pixels (they aren't pure black i.e. no NoData values so they should display) are shown white. If I convert the JPEG to a TIFF in IrfanView then use FME to load that into an FGDB raster it displays fine.

Another thing I think I might need to do is convert the source TIFFs to 3 band RED8, GREEN8, BLUE8 so that they go into the FGDB raster like this which enables better looking display in ArcMap. I think I need to use the RasterBandAdder for this but need a bit of help...

OK. I've found out why the image isn't displaying in ArcGIS correctly. It's because the Blue band has "NoData" as it's value for those pixels. I've only done a quick spot check, but I haven't found any other bands with "NoData" values, just the Blue one. I have no idea how this has happened.

 

To expand on my need to increase the source TIFFs to 3 band RED8, GREEN8, BLUE8... This is because some of them will be merged with JPEGs with a higher colour depth in the mosaicked raster.

 

"NoData" seems to be a key issue here which I think I've now fixed for the source TIFFs. I use the RasterBandNoDataRemover, then expand the source TIFFs to 3 band RED8, GREEN8, BLUE8 with the RasterPaletterResolver, then use the RasterExpressionEvaluator to subtract 1 from the pixel value. So 255 becomes 254 but 0 stays as 0. Then I can use the RasterBandNoDataSetter to set the NoData value to 255. However, after the Clipper, I now get "2017-08-16 19:08:28| 17.1| 3.1|WARN |TIFF writer: One or more pixels clipped to fit 1 bit domain." To be continued...

 


Maybe if you use the RasterCoercer to set everything to RGB24?


Maybe if you use the RasterCoercer to set everything to RGB24?

Thanks. I'm sure I tried that early on but I've learnt so much about rasters and processing them in FME since that it's probably worth looking again.

 

 


I've attached my current process for TIFFs which isn't working. I get a corrupt TIFF out at the end and FME logs this warning: "TIFF writer: One or more pixels clipped to fit 1 bit domain".

The source TIFFs are 1 bit CCITT T.6 compressed. I have to use the RasterPaletteResolver to remove the palette otherwise I can't use the RasterInterpretationCoercer to convert it to RGB24.

The FeatureMerger joins georeferencing data from a CSV which is then used by the RasterGeoreferencer. I'm using the RasterBandNoDataSetter because the pixel values of the data are now 0 and 255, and I want to keep the white background.

I have previously used RasterExpressionEvaluator to adjust the pixel values (e.g. A[0]-1) so that there's no conflict between data and NoData. I've also tried doing the conversion to RGB24 after the Clipper but haven't had any success with that either.


I've attached my current process for TIFFs which isn't working. I get a corrupt TIFF out at the end and FME logs this warning: "TIFF writer: One or more pixels clipped to fit 1 bit domain".

The source TIFFs are 1 bit CCITT T.6 compressed. I have to use the RasterPaletteResolver to remove the palette otherwise I can't use the RasterInterpretationCoercer to convert it to RGB24.

The FeatureMerger joins georeferencing data from a CSV which is then used by the RasterGeoreferencer. I'm using the RasterBandNoDataSetter because the pixel values of the data are now 0 and 255, and I want to keep the white background.

I have previously used RasterExpressionEvaluator to adjust the pixel values (e.g. A[0]-1) so that there's no conflict between data and NoData. I've also tried doing the conversion to RGB24 after the Clipper but haven't had any success with that either.

I've read these two articles but am still not sure what to do. Should I be using an Alpha band?

 

https://knowledge.safe.com/articles/19620/raster-clipping-1.html

 

https://knowledge.safe.com/articles/18764/setting-alpha-vs-nodata-color-and-numeric-data.html

 


I've attached my current process for TIFFs which isn't working. I get a corrupt TIFF out at the end and FME logs this warning: "TIFF writer: One or more pixels clipped to fit 1 bit domain".

The source TIFFs are 1 bit CCITT T.6 compressed. I have to use the RasterPaletteResolver to remove the palette otherwise I can't use the RasterInterpretationCoercer to convert it to RGB24.

The FeatureMerger joins georeferencing data from a CSV which is then used by the RasterGeoreferencer. I'm using the RasterBandNoDataSetter because the pixel values of the data are now 0 and 255, and I want to keep the white background.

I have previously used RasterExpressionEvaluator to adjust the pixel values (e.g. A[0]-1) so that there's no conflict between data and NoData. I've also tried doing the conversion to RGB24 after the Clipper but haven't had any success with that either.

RasterPaletteResolver gives me 3 bands so I'm not sure the RasterInterpretationCoercer is necessary.

 

 


Thanks to Dave Campanas at Safe (@DaveAtSafe?) for getting me to the solution for TIFFs:

"This is a bit tricky. The settings from the incoming Tiff
data are being propagated to the output through the format attributes on the
source image. These attributes override the bits per cell and compression set
on the output, leading to the 1 bit output. These format attributes are
designed to make TIFF to TIFF conversions simpler, but do assume that the
output structure matches the input.



I have added a BulkAttributeRemover to your workspace to
remove these attributes, and have also unset the bits per pixel setting in the
output feature type so that it will default to the standard 8 bits per band.
The output now appears correct."

(I had been trying the bits per pixels with and without values).


JPEG issue solved :-)

Some pixels have a value of zero in one band (often the blue band it seems), which is the NoData value. It seems that even though other bands have non-zero values in them, the NoData value in the blue band overrides them. So an RGB value of 40,37,0 will get displayed (or rather not displayed!) as NoData, rather than the dark grey it is meant to be. I assume this is expected behaviour...?

Anyway, my fix for this is the same as for the 1-bit TIFFs. I use the RasterExpressionEvaluator to increment the values in each band by 1. Then I can use the RasterBandNoDataSetter to set the NoData value to zero. It would be nice to only do this for pixels with a value of zero but it's not crucial.


Reply