Skip to main content

Hi all,

I am almost where I want to be with my FME script, however am encountering a road block as the final portion of the script is writing duplicate records and adding geometry to my target feature class, when I want the original records in the target feature class updated.

I have two separate polygon feature classes, one entitled TDE Projects and another entitled TPL Corridors and Concepts.

There is one Project Number (text) attribute that only exists in the TPL Corridors and Concepts layer that I want translated into overlapping polygons where TDE Projects and TPL Corridors and Concepts polygons overlap and have the Project Number from the TPL Corridors and Concepts layer inputted into the TDE Projects records.

Right now, my script is performing the overlay correctly and inputting the Project Number into a new field I created in the TDE Projects table entitled "TPL Project Number", however it is also generating duplicate records/geometry.

Ex:

 

Correct record in TDE Projects with the corresponding TPL Project Number:

image.png 

Duplicate record in TDE Projects without the TPL Project Number:

image 

Just wondering if you could give input on if there is a way to only have one record/geometry for overlapping TDE Project polygons. The script seems to be adding a new polygon to the TDE Projects layer and appending a TPL Project Number to it, but leaving the original record’s TPL Project Number blank. I don’t want a new geometry/record created, just to have the TPL Project Number translated to the original TDE Projects polygon that overlays it.

 

imageimage

Hi,

If I understand your workflow correctly, you are trying to the projectnumber attribute from TPL_Corridors_and_Concepts feature type to the TDE_Projects_AC feature type?

If this is the case, I don't believe that it is the AttributeManager that is causing an issue, but rather the AreaOnAreaOverlayer.

 

The AreaOnAreaOverlayer will take both feature types, and intersect them with one another. Attributes from TPL_Corridors_and_Concepts will be joined to TDE_Projects_AC and vice versa. That is where the extra geometries are coming from - where the TPL_Corridors_and_Concepts have tried to get a join from TDE_Projects_AC. This is also why those features have no project number in your output.

 

I would suggest using the SpatialRelator transformer instead of the AreaOnAreaOverlayer transformer, with TDE_Projects_AC as Requestor and TPL_Corridors_and_Concepts as supplier. This will mean that only the TDE_Projects_AC features will look for a join from the other feature type. Make sure that you tick Merge Attributes:

imageYou may have to tinker with the Spatial Predicates to Test option to identify whether the output is working as expected. You may also want to choose Generate List to create a list of projectnumbers if the TDE_Projects_AC features are likely to intersect multiple different TPL_Corridors_and_Concepts polygons.

 

This should give you a starting point:

imageThis should mean that duplicates aren't written, and hopefully remove some the redundancy that appears to be in your workspace.


Hi @churchfraser​ , thanks so much for your response and the screenshots!

 

It definitely helped removed a significant amount of the redundancy in my workspace. I've created a workspace as specified and used the Spatial Relator transformer instead - thanks so much again.

 

I threw the results into a web map, and it seems as if there are some duplicate records still. The attribute table for the original feature class had 270 records, and once completed, it has 313 records.

 

Ex:

OBJECTID 216 is the original record in the table with no TPL Project Number Populated

imageOBJECTID 298 is an exact duplicate of the same project, but with the TPL Project Number correctly added in

imageThe only item that differs between the 2 duplicates is the Global ID number. I was curious if there was a way to update the original record, in this case, OBJECT ID 216 with the TPL Project Number

This occurs for all of the TDE Projects that interesect with the TPL Corridors and Concepts (around 44 records).

Is it a setting in my final writer that should be adjusted perhaps? I notice the default option for Feature Operation is Insert and Create if Needed. I've tried both Insert, Create if Needed as well as Insert, Use Existing, with the same result. However, when I select the Update option I get the mandatory field for Row Selection, Match columns which I am not sure about. Is this Match columns field a unique field to select from my TDE Projects layer?

imageThanks so much in advance


You're right, it is the setting in the final writer that need to be updated. You've got the settings there almost right. I would suggest

  • Feature Operation = Update
  • Table Handling = Use Existing
  • Update Spatial Columns = Yes
  • Match Columns = ObjectID

The Match Columns can be set to any column(s) that are unique (or a unique combination) when writing to SDE or Geodatabase. ObjectID is easiest, as this is guaranteed to be unique because of the format.

Note that if you eventually end up reading from & writing to an Esri ArcGIS Online or Portal feature service, the default Match Column is the ObjectID (with option to set to GlobalID). Setting up your workspace with this in mind should help head off any potential issues.

 

You will unfortunately have to manually remove any duplicates that have been inserted into the table from testing. Going forward, this shouldn't be a requirement.

 

I am assuming that your workflow will always read from & write to the same feature type. But if you are reading from one dataset and writing to another, you may want to use a ChangeDetector transformer to identify updates and expose the ObjectID of those features when writing.


You're right, it is the setting in the final writer that need to be updated. You've got the settings there almost right. I would suggest

  • Feature Operation = Update
  • Table Handling = Use Existing
  • Update Spatial Columns = Yes
  • Match Columns = ObjectID

The Match Columns can be set to any column(s) that are unique (or a unique combination) when writing to SDE or Geodatabase. ObjectID is easiest, as this is guaranteed to be unique because of the format.

Note that if you eventually end up reading from & writing to an Esri ArcGIS Online or Portal feature service, the default Match Column is the ObjectID (with option to set to GlobalID). Setting up your workspace with this in mind should help head off any potential issues.

 

You will unfortunately have to manually remove any duplicates that have been inserted into the table from testing. Going forward, this shouldn't be a requirement.

 

I am assuming that your workflow will always read from & write to the same feature type. But if you are reading from one dataset and writing to another, you may want to use a ChangeDetector transformer to identify updates and expose the ObjectID of those features when writing.

Hi @churchfraser, that worked!

The switching of the transformers from AreaOnAreaOverlayer to the SpatialRelator kept the duplicate geometries from writing into the final TDE Projects layer, while specifying the Feature Operation to 'Update' with Table Handling set as 'Use Existing' and Match Columns with 'ObjectID' ensured that the only relevant records were updated with the relevant TPL Project Numbers and no duplicate records were inserted.

Now I'll play around with the Generate List to create a list of projectnumbers since you are correct in pointing out that several of the TDE_Projects_AC features are likely to intersect multiple different TPL_Corridors_and_Concepts polygons. Then I'll research and play around with the relationship classes and many to many cardinality along with relates to see if I can finalize this, to ensure that relates display multiple TPL Corridors and Concepts when applicable.

Thanks so much for taking the time to explain so clearly along with screenshots. I'm new to FME and this definitely helped me out big time.

 


Hi @churchfraser, that worked!

The switching of the transformers from AreaOnAreaOverlayer to the SpatialRelator kept the duplicate geometries from writing into the final TDE Projects layer, while specifying the Feature Operation to 'Update' with Table Handling set as 'Use Existing' and Match Columns with 'ObjectID' ensured that the only relevant records were updated with the relevant TPL Project Numbers and no duplicate records were inserted.

Now I'll play around with the Generate List to create a list of projectnumbers since you are correct in pointing out that several of the TDE_Projects_AC features are likely to intersect multiple different TPL_Corridors_and_Concepts polygons. Then I'll research and play around with the relationship classes and many to many cardinality along with relates to see if I can finalize this, to ensure that relates display multiple TPL Corridors and Concepts when applicable.

Thanks so much for taking the time to explain so clearly along with screenshots. I'm new to FME and this definitely helped me out big time.

 

Hey @alexcua87,

Happy to help out, welcome to FME!

For the Generate List function, you could use a ListConcatenator after the SpatialRelator. It concatenates the relationship attributes with whatever your chosen separator character is (including none or space). So if a project from TDE_Projects_AC intersects two features from TPL_Corridors_and_Concepts, both those values will be concatenated into one attribute.

The example below separates the values using a semi-colon (;)

image.png 

It might be worth considering the intersections that you actually want to have displayed. I believe the spatial predicate I've set above will show intersections where boundaries intersect one another, not necessarily 'real' relationships.

You could do something like reduce the size of TDE_Projects_AC using a negative buffer, spatially relating these, and joining the relationship back onto the original geometry (as writing the buffered geometry using UPDATE will replace the original geometry).


Hi @churchfraser​ !

Thanks so much for the suggestion. Definitely works as expected!

I will play around with the spatial predicate but this is a great start on getting the intersecting relationships displayed in a clear manner.

I'll need to play with some relationship classes in ArcGIS Pro now, as we want to have the related records with multiple intersections show up in the related records tab so will move to that area now.

Thanks so much again for explaining everything so clearly and concisely!


Reply