Skip to main content
  • EN
Hi

 

 

I'm looking for a quick way (hopefully) to identify fields that will have their values truncated on writing to output. Using the StringLengthCalculator works great but only on a single field, so I would need to string lots of them together - as my data is being written to a database with multiple tables and multiple attributes per table, I'm trying to avoid this. I need something like the StatisticsCalculator which can generate summary stats for multiple fields at a time,.. but the problem being calculating the string length first as StatsCalc works on the values. And my output values will differ to input values.

 

I expect this is a comon problem - is there something I'm missing?

 

Any ideas & help much appreciated,

 

 

Thanks

 

Mary
Hi,

 

 

this sounds like a case where a custom transformer could be in order. This way you could re-use the functionality several times over in your workspace(s).

 

 

The challenge is that the workspace transformers doesn't have any knowledge about the output format at run-time, meaning that your (custom) transformer won't know the field length limitations in your output feature.

 

 

You could define the maximal field lengths of the output feature types in an external document (e.g. csv or Excel) and then use the SchemaMapper from your custom transformer to compare and flag as appropriate.

 

 

David
Hi,

 

It is indeed a common problem and even the store transformer (StringLengthFormatter) only works with one attribute at a time.

 

 

Hi,

 

 

What I did in the past was to export the db schema, that way you know the destinations attributes lenghts and use that in the ws to compare to the input attributes.

See this...

https://knowledge.safe.com/questions/51855/how-to-find-truncated-values.html

But more importantly, vote for this idea!

https://knowledge.safe.com/content/idea/75676/enhanced-logging-of-data-truncation-warnings.html

Reply


Community Stats

31,758
Posts
120,645
Replies
39,383
Members

Latest FME

Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.