Skip to main content
Question

Attribute Data Type Preservation in FME 2020


mark2atsafe
Safer
Forum|alt.badge.img+43

Hello FME users,

I just wanted to share some information about an update to data types in FME2020, which I don't think has been publicized much. The best demonstration is through a video, but first here's a brief description of the update...

Data Type Preservation

In previous versions of FME, attribute data type would not be preserved fully throughout a workspace. For example, a writer could not determine the data type of an attribute added or edited in the workspace.

You could prove this by following these steps:

  • Open FME Workbench 2019
  • Add a writer and set the feature type attributes to automatic
  • Add an AreaCalculator transformer connected to the writer

The result is that _area is added to the writer, as type char(200). The writer was not able to tell that this attribute should be a numeric type. You would often see char(200) fields as a default for unknown types.

In FME 2020 this has changed.

If you repeated the above step in FME 2020, the result would be a real64 field (or the equivalent of that for the format being used).

In short, data types are now better preserved throughout a workspace.

Video Demonstration

Here's a video demonstration of the improved functionality in FME 2020

Note: If you don't see the video above, it's attached here:

 

Summary

I hope you find this of use. You don't need to change any settings to apply this; you'll just find that FME is doing a better job at handling data types throughout a workspace.

9 replies

bwn
Evangelist
Forum|alt.badge.img+26
  • Evangelist
  • April 17, 2020

That's a nice improvement @mark2atsafe . I also found that previously simply renaming an Attribute also caused issues so I'm guessing that is handled as well?


itay
Supporter
Forum|alt.badge.img+16
  • Supporter
  • April 17, 2020

indeed a great addition @mark2atsafe, does it also apply to automatic attribute definition?


mark2atsafe
Safer
Forum|alt.badge.img+43
  • Author
  • Safer
  • April 17, 2020
bwn wrote:

That's a nice improvement @mark2atsafe . I also found that previously simply renaming an Attribute also caused issues so I'm guessing that is handled as well?

Yes, absolutely.


mark2atsafe
Safer
Forum|alt.badge.img+43
  • Author
  • Safer
  • April 17, 2020
itay wrote:

indeed a great addition @mark2atsafe, does it also apply to automatic attribute definition?

I think so, yes.


fmeuser_gc
Contributor
Forum|alt.badge.img+8
  • Contributor
  • April 19, 2020

Nice to see this feature @mark2atsafe. But it looks like Attribute field 'Names' are not being preserved (truncated when compared with Geodb reader) in shapefile writer for few of the attributes. Any thoughts on this?


bwn
Evangelist
Forum|alt.badge.img+26
  • Evangelist
  • April 20, 2020
fmeuser_gc wrote:

Nice to see this feature @mark2atsafe. But it looks like Attribute field 'Names' are not being preserved (truncated when compared with Geodb reader) in shapefile writer for few of the attributes. Any thoughts on this?

Many Destination formats can't support particular Field Name formats, Shape files and the associated dBase files being notable examples due to their ~40 year old architecture.

That isn't a limitation on FME but a limitation of the Writer Destination Format.


itay
Supporter
Forum|alt.badge.img+16
  • Supporter
  • April 20, 2020
mark2atsafe wrote:

I think so, yes.

Yes! can confirm that it does.


mark2atsafe
Safer
Forum|alt.badge.img+43
  • Author
  • Safer
  • April 20, 2020
fmeuser_gc wrote:

Nice to see this feature @mark2atsafe. But it looks like Attribute field 'Names' are not being preserved (truncated when compared with Geodb reader) in shapefile writer for few of the attributes. Any thoughts on this?

Yes, as @bwn mentions, Shapefile - because it uses dBase - has a field name length limit of 10 characters. FME automatically truncates names where necessary to fit this requirement.


That is, indeed, an appropriate and convenient change, but I find that it is not actually fixed, at least in my installation of FME 2020.1 Desktop Professional. Now, there may be a patch or the like that didn't get installed in my case, or, after my Windows Profile meltdown and rebuild a few weeks ago, that was "lost track of" in some way. I don't know; however, my 2020.1 is still dropping the attribute type info on write. ( ... and yes, I can and do make the very easy changes necessary by switching User Attributes in the writer to Manual and changing the attribute types there.)


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings