Skip to main content
Open

Introduce a manual equivalent to SchemaScanner (SchemaCreator)

Related products:Transformers
  • July 3, 2024
  • 1 reply
  • 37 views
vlroyrenn
danilo_fme
warrendev
  • vlroyrenn
    vlroyrenn
  • danilo_fme
    danilo_fme
  • warrendev
    warrendev

vlroyrenn
Supporter

Context: Schema features

When dealing with dynamic workflows, FME introduces the need to work with schema features, which are used by some transformers to recognize all the “columns” (attribute types, list attributes, struct fields with their types, geometry type, etc.) to be found in incoming features. SchemaScanner is able to generate those from data of an unknown type by (as the name suggest) scanning the first N lines of data and assuming it is representative of the rest. Dynamic input nodes for Custom Transformers also can’t know the schema of incoming features unless a schema feature is supplied.

The Problem: Making/getting schema features (when you know your schema)

The thing is, even in dynamic workspaces, users tend to have some notion of what their data format is at any given point (as can be previewed by looking at cached features in FME Form, and especially if AttributeManager is used to set attribute types), yet getting a schema feature representative of that known format is fairly difficult. Official tutorials reccomend making one by hand, using an AttributeCreator, which basically achieves the same result as the “User Attributes” tab of a FeatureWriter, only much more clumsily and difficult to keep up to date when attributes are added or removed. FME deliberately avoids exposing enough information to bridge that gap in Python or something similar.

The Solution: SchemaCreator?

FME should probably have a transformer that’s a cross between a SchemaScanner and a FeatureReader: a transformer that recieves features and outputs a single schema feature, before reoutputting the input features. Unlike SchemaScanner, though, the schema definition would be static, defined by the user (or detected automatically) using a Feature Type menu mostly identical to what is used by FeatureWriter, with the cached input feature values used to provide type suggestions.

A mock-up of the basic idea. The interface only needs to reproduce the Feature Type menu used by writers and FeatureWriter, and for the transformer to output it as-is on the <Schema> port.

 

1 reply

LizAtSafe
Safer
Forum|alt.badge.img+15
  • Safer
  • August 15, 2024
NewOpen

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings