Enhanced logging of data truncation warnings

Yes please! This is so annoying! I've had this problem a few times, but now I've got it for a dataset of 40 million records X 128 fields. I've got a great set of transformers (AttributeExploder > AttributeCreator (StringLength of each _attr_value) > StatisticsCalculator to get maximum) that will give me the max length of every value in every field, but I think my computer would melt if I throw that much data at it!

Could the FeatureWriter be enhanced to output features where attributes were truncated through a separate port, or add attribute(s) to each feature to say if values were truncated?

I'm seeing this with the Geodatabase Writer and it would be very helpful to have this information so we could see if it's an issue we need to address or not.

The level of detail you get with truncate warnings seems to vary depending on the format being written, e.g. shape file provides the truncated value in the log (although not what field it's in), writing to ArcSDE and you get "Geodatabase Writer: 3 attribute value(s) were truncated" only.

I've just written a little bit of python that will check the string lengths based on a schema list which at least avoids exploding all the data (the schema data needs to be hardcoded at the moment)

As an aside, does anyone know when truncate warnings were introduced, or have they always been logged? @mark2atsafe

This is still an irritation for me. If I knew which fields the values were truncated in, I could make a judgement call about whether they were important or not.

Hello,

to add my two cents, this very important enhancement request, when converting big bunch of files (in my case more than 1 million of XMLs to Esri File Geodatabase), it would be more than helpful to see which features were affected - and how much, or at least which fields and how much (in maximum), to I can change the setting of the output field appropriately!

Thanks.

Agree with this.  Any action that changes data should be clearly logged and even added as a warning message at the end of the translation - ‘Translation Successful with Warnings’ for truncation and possibly other warnings - as an option.