Skip to main content
Question

Defines the short type but generates the long type (Shape Writer)

  • October 12, 2017
  • 7 replies
  • 127 views
taojunabc
Participant
Forum|alt.badge.img+6

FME 2017.1.1.0

I manually defined in the shape writer a field type of short, when the data is generated, In ArcMap, the field type shown as Long. Is this a bug?

By the way, I define a type of number, width of 4,the fractional part width of 0?

in ArcMap shows the field type is Short.

It seems that FME 2016 has the same problem, but it seems still not being resolved.

https://knowledge.safe.com/questions/32929/short-datatypes-understood-as-long-with-new-shapef.html

7 replies

nathanatsafe
Safer
Forum|alt.badge.img+7

Hi @taojunabc,

Thanks for bringing this issue to our attention. You're right to point at that this problem has been around for a while - and is a tough one to get right. The issue of preserving numeric type field widths when reading and writing Shapefiles is being worked on by development currently, and, if you are able to download and try out our latest beta, you may already see a difference in behaviour.

Hope this helps!

Nathan

taojunabc
Participant
Forum|alt.badge.img+6
  • taojunabcParticipant
  • Author
  • Participant
  • October 13, 2017
nathanatsafe wrote:

Hi @taojunabc,

Thanks for bringing this issue to our attention. You're right to point at that this problem has been around for a while - and is a tough one to get right. The issue of preserving numeric type field widths when reading and writing Shapefiles is being worked on by development currently, and, if you are able to download and try out our latest beta, you may already see a difference in behaviour.

Hope this helps!

Nathan

Hi @NathanAtSafe

 

Thanks for your reply. I will confirm the latest beta when I am free.

 

 

Now, when I manually add attributes, because I know the problem, I can set the type number: 4:0.

 

However, I often import the FeatureTypes from the existing Shapefiles, and the short type will be set to the short type by default. If I forget to modify type, the attributes of the converted data will be displayed as long in ArcGIS.

 

takashi
Influencer
  • takashiInfluencer
  • October 13, 2017

I cannot see any improvement in the latest 2018 beta build 18185.

b18185 Shapefile Writer Feature Type Setting

b18185 Shapefile Reader Feature Type configured from dataset written by the Writer above

I think this is the correct schema - FME 2015.1.3.2 build 15575 Shapefile Reader Feature Type.

M
takashi wrote:

I cannot see any improvement in the latest 2018 beta build 18185.

b18185 Shapefile Writer Feature Type Setting

b18185 Shapefile Reader Feature Type configured from dataset written by the Writer above

I think this is the correct schema - FME 2015.1.3.2 build 15575 Shapefile Reader Feature Type.

To preserve existing functionality since FME 2016, we added a new option to the Esri Shapefile reader to explicitly enable the number(x,y) type behaviour. To get the new functionality, ensure you change the new Numeric Attribute Type Handling parameter in the reader options to Explicit Width and Precision. This will produce schema that matches the attribute widths in the DBF file.

 

 

takashi
Influencer
  • takashiInfluencer
  • October 14, 2017
takashi wrote:

I cannot see any improvement in the latest 2018 beta build 18185.

b18185 Shapefile Writer Feature Type Setting

b18185 Shapefile Reader Feature Type configured from dataset written by the Writer above

I think this is the correct schema - FME 2015.1.3.2 build 15575 Shapefile Reader Feature Type.

I checked the new parameter "Numeric Attribute Type Handling" in the Shapefile reader, FME 2018.0 beta build 18185.
  • Actually the Shapefile format doesn't support the numeric data types (short, long, float, and double). Those are just aliases of specific "number(p, w)", and their definitions are vary depending on the context. Even in ArcGIS, the definitions could be different depending on versions. I therefore think the native data type "number(w,p)" should be considered as the standard (default) and the fake data types should be an option.
  • Writer: If you intend to map 32 bit integer to the fake long, corresponding native data type should be "number(11, 0)", because the range of 32 bit integer is -2,147,483,648 (11 digits) .. 2,147,483,647.
  • Reader: The native data type "number(10, 0)" should not be mapped to actual long (32 bit integer), because the range of "number(10, 0)" (-999,999,999 .. 9,999,999,999) could overflow the range of 32 bit integer (-2,147,483,648 .. 2,147,483,647).

 

takashi
Influencer
  • takashiInfluencer
  • October 14, 2017
Hi @taojunabc, the Shapefile format only has the decimal numeric data type "number(w, p)", and doesn't support standard numeric data types such as short, long, float and double. Those data type names for the Shapefile format used in an application are the aliases of specific "number(w, p)", and their definitions may be different depending on applications.

 

If you set "short" to the data type of a field with FME Shpefile writer, it would be interpreted as a specific "number(w, p)" - maybe "number(6, 0)" by FME, but ArcGIS interpreted it as "long". I think that's the situation you have observed.

 

It's just a difference in the interpretation on the fake data types between applications, so I don't think it's a bug.

 

taojunabc
Participant
Forum|alt.badge.img+6
  • taojunabcParticipant
  • Author
  • Participant
  • October 16, 2017
takashi wrote:
I checked the new parameter "Numeric Attribute Type Handling" in the Shapefile reader, FME 2018.0 beta build 18185.
  • Actually the Shapefile format doesn't support the numeric data types (short, long, float, and double). Those are just aliases of specific "number(p, w)", and their definitions are vary depending on the context. Even in ArcGIS, the definitions could be different depending on versions. I therefore think the native data type "number(w,p)" should be considered as the standard (default) and the fake data types should be an option.
  • Writer: If you intend to map 32 bit integer to the fake long, corresponding native data type should be "number(11, 0)", because the range of 32 bit integer is -2,147,483,648 (11 digits) .. 2,147,483,647.
  • Reader: The native data type "number(10, 0)" should not be mapped to actual long (32 bit integer), because the range of "number(10, 0)" (-999,999,999 .. 9,999,999,999) could overflow the range of 32 bit integer (-2,147,483,648 .. 2,147,483,647).

 

@takashi

 

Thank you for your investigation, it seems that due to the definition of Shape (DBF) itself and the description of ArcGIS, it is not so easy to solve this problem. The use of number (x, y) to represent the form of data is accurate, but for ArcGIS users, may not be particularly easy to understand.

 

 

I think that as an alternative, FME can add an ArcGIS Vision parameter in the Shape Reader and Shape Writer. When user select a version, the data types for Short, Long, etc. are set later, and the internal version is automatically switched according to the version. The width and precision of the type number. In the strict sense, this method is not so tight, However, if the definition of FME in the set of ArcGIS version of the case, and ArcGIS description can be consistent, then in most cases may be less to the user confusion.

 

Reply


Helpful Members This Week

Community Stats

31,183
Posts
118,189
Replies
38,791
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.