I have 2 datasets within my workspace and wish to append the contents from one into the other. Will need to map fields as they are not named the same. (or can do an attributerenamer step beforehand. ) No join clause is required, it’s just a simple append.
What are the recommended steps?
A core concept in FME is that there are no “tables” in an FME workspace so there is no “append”.
FME processes work on a feature-by-feature basis where conceptually each feature is an individual element that is independent of all other features. For example 100 Features created by Reader A, 300 Features created by Reader B, and, and 10 Features created by a Creator etc. are 100+300+10 = 410 individual features that do not belong to any workspace table or dataset. If all 410 features were routed to the input Port to a single Transformer or Writer then the Transformer/Writer doesn’t care where they came from, it will process them one-by-one independently**
**A caveat is on how some FME Transformers create and handle features placed into “bulk mode”, which in their design and behavior are analogous to whole “tables” of data, but that is a more advanced topic.
Generally “best practice” is to, immediately following the Reader(s) to change the Attribute Names to the desired workspace schema names, preferably the final Writer field names where possible. The most versatile Transformer for this is AttributeManager which allows multiple schema mapping operations to be done at once: Rename Attributes, Reorder Attributes, Create New Attributes, Remove Attributes, Expose any currently Unexposed Attributes etc. If done at the very start of the workspace, usually immediately following the Reader(s) this makes it generally easier when routing these features into downstream Transformers and Writers to have consistent, expected feature-by-feature processing behavior as each feature enters them. Once the attribute names and desired attribute order has been set, then the Features from each of these AttributeManagers can be routed into a common Transformer or Writer which is the closest approximation to “appending” one dataset to another, even though that is not what is really happening in the workspace since the downstream Transformer/Writer processes on a feature-by-feature basis (excepting when the data enters in bulk mode to a bulk mode enabled Transformer as above).
This is not compulsory however. It is possible to leave Dataset A’s attributes and Dataset B’s attributes with their original names, up to and within the Writer(s) since most Writers have the ability to also manually map Feature Attribute Names to Destination Fields and can direct multiple differently named Attributes to a single destination field to perform the workspace->destination field mapping this way. It is just that this is often “harder” to design a workspace this way.
Great answer thank you
I wasn’t sure I could add multiple inputs to a writer which was why I wondered about appending first.
But it seems I can so therefore, the best approach in my case would be to rename all attributes involved before sending directly to the writer so they all match.
I wasn’t sure I could add multiple inputs to a writer which was why I wondered about appending first.
As someone who moved to FME several years ago from a background of ArcGIS ModelBuilder scripting where “Append” was a heavily utilised ArcTool in the ArcGIS Scratch Workspace: Don’t worry, was perhaps my very first FME question I had on what was the FME equivalent 