Question

I want the workflow to fail when an attribute is removed from the source table

  • 8 March 2018
  • 1 reply
  • 1 view

We’re doing nightly imports from various sources into PostgreSQL. Right now we aren’t using FME for these imports, but are in the process of switching to FME.

The PostgreSQL tables have fixed “schemas”/table definitions and since there are views using the tables they cannot be recreated. So they are truncated before new data is inserted.

The source may for example be an Oracle-table. We import a subset of the columns into a PostgreSQL table. Some column names may change, often we add a sequence and we may also change e g geometry type. There may be other transformations as well.

If a column, that is to be imported into the target PostgreSQL-table, gets removed we want the import to fail and the target table to be left untouched. This also if the column isn’t set to mandatory in the PostgreSQL database. So far the only solution I’ve found is using an AttributeValidator to verify that the attribute exists. Without this validator the workflow is carried out without errors and the column removed from Oracle gets null values in all PostgreSQL-table-records. Using an AttributeValidator isn’t really a perfect solution since there are a lot of columns in many tables and we have a lot of tables to import,


1 reply

Userlevel 6
Badge +32

One way to do this is to read the the source and target schemas and compare those. Only if source equals target fire the translation. You might want to stage that if you need a generic solution. Let the first workspace compare the schemas and fire the second workspace with a workspacerunner to do the job if the schemas really match.

I do this with a FeatureReader with the parameter Schema Features.

Reply