Question

Deleting a value that does not exist in a coded domain?

E

Hi all,

I am a newer user to FME, and I am using FME Workbench 2017.0.

I have a new feature class that has coded domains. The data being input into the new feature class is coming from a feature class in an old MDB. I would like to delete attribute values from each field that does not have values within their respective coded domains, and they can either be null or empty strings.

The picture shows an example of some of the data where you can see BUR-T does not exist as a coded value in responsible_agency or maintained_by, but it exists in pipe_owner. Or STR does not exist as a coded value in pipe_owner or maintained_by, but it exists in responsible_agency. I used AttributeValueMapper to get to this point.

Note: I have tried making a query to delete values with char_length < #, deleting uppercase values, and etc., and those did not seem to work because the 'Town of Burke' and 'Streets Division' values are still recognized as their codes, so they are subsequently deleted when the program is executed.

Any help would be appreciated. Thank you!

8 replies

takashi
Rank
Userlevel 2
Badge +17
I cannot understand what "coded domain (value)" is. Can you consider "a string consisting of uppercase and hyphen only" as "coded domain"?

 

Can you post your desired result table corresponding to the source table you have posted above?

 

chrisatsafe
Rank
Badge +2

Hi @enasb,

I created a sample workspace (coded-domain-example.fmwt) that might help you accomplish this task. Since you know the values you want to remove, you can use a conditional statement in the AttributeManager to re-assign values. For example, in the workspace I provided, I am testing for the value "StrongStart BC" in the school_category attribute. If a feature's school_category = "StrongStart BC" the AttributeManager will set the school_category to be empty (see the screenshot below for reference).

Since this was a simple example I used the exact spelling for the test clause; however, you can also use a Regular Expression in the test clause by changing the Test Clause operator to "Contains Regex".

To set up the conditional statement's test clause: Open the AttributeManager Parameters > Attribute Value (drop down arrow) > Conditional Value.. > Enter a Test Clause (double click empty cell) into the first "If" row and leave the Attribute Value blank.

It may be helpful to refer to the Testing with If -Then Conditional Statements article as it goes over several scenarios and into greater detail on how to set conditional actions with the AttributeManager (with very helpful screenshots).

I used the Attribute Manager because it keeps it all contained in a single transformer but you can also do this using other Transformers. Be sure to check out the short video on the Control Your Data's Attributes with the AttributeManager Transformer if you want a quick intro to the AttributeManager and the Working with Geodatabase Subtypes and Domains article too!

Hope this helps.

- Chris

E
I cannot understand what "coded domain (value)" is. Can you consider "a string consisting of uppercase and hyphen only" as "coded domain"?

 

Can you post your desired result table corresponding to the source table you have posted above?

 

Takashi,

 

 

I created a coded domain in ArcCatalog as part of a geodatabase, so FME would identify the input (TE, STR, BLO-T) and give me an output that correlates to the description of the code (Traffic Engineering, Streets, Town of Blooming Grove). Attached is an example of the one I am currently working with:

 

 

 

And the current issue with trying to identify characters such as "a string consisting of uppercase and hyphens" is the value "Town of Blooming Grove" appears to me as "Town of Blooming Grove" but it exists within the geodatabase as "BLO-T". But here is my desired results table where the empty strings could also be <Null> if that was the only way I could get it to work:

 

 

E

Hi @enasb,

I created a sample workspace (coded-domain-example.fmwt) that might help you accomplish this task. Since you know the values you want to remove, you can use a conditional statement in the AttributeManager to re-assign values. For example, in the workspace I provided, I am testing for the value "StrongStart BC" in the school_category attribute. If a feature's school_category = "StrongStart BC" the AttributeManager will set the school_category to be empty (see the screenshot below for reference).

Since this was a simple example I used the exact spelling for the test clause; however, you can also use a Regular Expression in the test clause by changing the Test Clause operator to "Contains Regex".

To set up the conditional statement's test clause: Open the AttributeManager Parameters > Attribute Value (drop down arrow) > Conditional Value.. > Enter a Test Clause (double click empty cell) into the first "If" row and leave the Attribute Value blank.

It may be helpful to refer to the Testing with If -Then Conditional Statements article as it goes over several scenarios and into greater detail on how to set conditional actions with the AttributeManager (with very helpful screenshots).

I used the Attribute Manager because it keeps it all contained in a single transformer but you can also do this using other Transformers. Be sure to check out the short video on the Control Your Data's Attributes with the AttributeManager Transformer if you want a quick intro to the AttributeManager and the Working with Geodatabase Subtypes and Domains article too!

Hope this helps.

- Chris

Chris,

 

 

I did try to use Attribute Manager with conditional statements and I haven't gotten it to work yet, but I noticed the reader field with the existing data populates all three fields, so it would delete the values that are valid:

 

 

 

Another issue, if I can get it to work, is that I would have to make one for all three fields (pipe_owner, responsible_agency, maintained_by) with about 25, 45, and 50 conditional values for each respective field. A regular expression would not work because it does not recognize that "Town of Blooming Grove" is different from "BLO-T" within the geodatabase, since it only exists as "BLO-T".

 

 

Although, if I can get it to work that way, I would also be very happy.

 

takashi
Rank
Userlevel 2
Badge +17

OK. If the destination format is Esri File Geodatabase, I know what the coded domain is.

As far as I know, FME cannot read the definition of domains from an FGDB dataset, but the GEODATABASE_FILE reader has an option parameter called "Resolve Domains" to read the description corresponding to each code. If you checked the option when adding the reader to a workspace, its reader feature type will have description attributes suffixed by "_resolved" for each attribute which has domain definition, and its value will be empty if the code isn't defined.

So, a workaround is: once write all MDB records into the destination FGDB table, re-read all records from the table using the GEODATABASE_FILE reader with the "Resolve Domains" option, replace codes that don't have description with <null> or <empty>, and then update the table.

This screenshot illustrates the processes from re-reading the table to updating it.

E

OK. If the destination format is Esri File Geodatabase, I know what the coded domain is.

As far as I know, FME cannot read the definition of domains from an FGDB dataset, but the GEODATABASE_FILE reader has an option parameter called "Resolve Domains" to read the description corresponding to each code. If you checked the option when adding the reader to a workspace, its reader feature type will have description attributes suffixed by "_resolved" for each attribute which has domain definition, and its value will be empty if the code isn't defined.

So, a workaround is: once write all MDB records into the destination FGDB table, re-read all records from the table using the GEODATABASE_FILE reader with the "Resolve Domains" option, replace codes that don't have description with <null> or <empty>, and then update the table.

This screenshot illustrates the processes from re-reading the table to updating it.

@takashi

 

Do you know if there's a way to automatically run the second workspace that doesn't use FME Server? I guess I don't know what the limitations are of FME, and if automating two workspaces and making sure the parameters of the second workspace are changed is too much to ask.

 

 

I have the pipes MDB --> AttributeValueMapper --> pipes FGDB in one workspace and pipes FGDB --> AttributeManager --> updated pipes FGDB in the other workspace.

 

takashi
Rank
Userlevel 2
Badge +17

OK. If the destination format is Esri File Geodatabase, I know what the coded domain is.

As far as I know, FME cannot read the definition of domains from an FGDB dataset, but the GEODATABASE_FILE reader has an option parameter called "Resolve Domains" to read the description corresponding to each code. If you checked the option when adding the reader to a workspace, its reader feature type will have description attributes suffixed by "_resolved" for each attribute which has domain definition, and its value will be empty if the code isn't defined.

So, a workaround is: once write all MDB records into the destination FGDB table, re-read all records from the table using the GEODATABASE_FILE reader with the "Resolve Domains" option, replace codes that don't have description with <null> or <empty>, and then update the table.

This screenshot illustrates the processes from re-reading the table to updating it.

I think you can create a workspace containing all the processes: Read MDB -> AttributeValueMapper -> FeatureWriter (FGDB) -> FeatureReader(FGDB, with Resolve Domains option) -> AttributeManager -> Update FGDB.

 

Alternatively, you can also use the WorkspaceRunner in the first workspace to kick off the second workspace: 1. Read MDB -> AttributeValueMapper -> FeatureWriter (FGDB) -> WorkspaceRunner (run the second workspace); 2. Reader FGDB -> AttributeManager -> Update FGDB.

 

 

E
I think you can create a workspace containing all the processes: Read MDB -> AttributeValueMapper -> FeatureWriter (FGDB) -> FeatureReader(FGDB, with Resolve Domains option) -> AttributeManager -> Update FGDB.

 

Alternatively, you can also use the WorkspaceRunner in the first workspace to kick off the second workspace: 1. Read MDB -> AttributeValueMapper -> FeatureWriter (FGDB) -> WorkspaceRunner (run the second workspace); 2. Reader FGDB -> AttributeManager -> Update FGDB.

 

 

I used FeatureWriter and FeatureReader and it worked like a charm. Thank you so much!

 

 

Reply


Community Stats

28,717
Posts
109,495
Replies
36,560
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.