Skip to main content
Solved

AttributeValidator Removing Record

mmccart
Contributor
Forum|alt.badge.img+16

I am using serveral AttributeValidator Transformers in FME Desktop 2019.1.2 and noticed that one of them is removing a record. I have a total of 48,328 records that I am trying to validate. However, the total output of Passed + Failed ports adds up to 48,327. The 3 AttributeValidators that I used upstream passed all records thru.

Screenshot of Workbench:

AttributeValidator Settings:

 

It appears that only the 4th AttributeValidator in the workbench does this. Is there a limit to the number of AttributeValidators that a single workbench can contain. I need to use multiple AttributeValidators as I have to perform 6 unique validation test.

Has anyone seen this behavior? If so, what is the fix/work-around? Or is there better way to validate data without using the AttributeValidator?

Thanks,

Mark

Best answer by mark2atsafe

The good news is the original issue is fixed. It's gone into various versions, so I'll list them all:

  • 2019.1: Build 19643
  • 2019.2: Build 19785
  • 2020.0: Build 20105

So check the version/build of the FME you are using and upgrade as necessary. 2019.2 is still in beta but I suspect just 2-3 weeks til it's released.

The not-so-good news is that I'd have to agree that the Tester is quicker at the moment. It's been updated to use "bulk mode" but the AttributeValidator has not. In time, that too will acquire bulk mode and be just about as quick as the Tester, but I don't know when that might be.

View original
Did this help you find an answer to your question?

10 replies

fmelizard
Contributor
Forum|alt.badge.img+17
  • fmelizardContributor
  • Contributor
  • October 7, 2019

Hi @mmccart This sounds like a problem to me. Please send details to https://www.safe.com/support/report-a-problem/ Thanks!

erik_jan
Contributor
Forum|alt.badge.img+17
  • erik_janContributor
  • Contributor
  • October 7, 2019

Have you tried connecting a next transformer (for example an Inspector) to the output?

I have noticed the count not finishing in some weird occasions.

Hope this is one of those.

mmccart
Contributor
Forum|alt.badge.img+16
  • mmccartContributor
  • Author
  • Contributor
  • October 7, 2019
erik_jan wrote:

Have you tried connecting a next transformer (for example an Inspector) to the output?

I have noticed the count not finishing in some weird occasions.

Hope this is one of those.

@erik_jan, I added an AttributeManager and an Inspector and the record still missing. Here are screenshots:

 

 

I'm going to submit a case to Safe per @danatsafe comment below.

erik_jan
Contributor
Forum|alt.badge.img+17
  • erik_janContributor
  • Contributor
  • October 7, 2019
mmccart wrote:

@erik_jan, I added an AttributeManager and an Inspector and the record still missing. Here are screenshots:

 

 

I'm going to submit a case to Safe per @danatsafe comment below.

And no indication why it skipped a feature in the log file?

In that case you should send a request to www.safe.com/support

mmccart
Contributor
Forum|alt.badge.img+16
  • mmccartContributor
  • Author
  • Contributor
  • October 7, 2019
fmelizard wrote:

Hi @mmccart This sounds like a problem to me. Please send details to https://www.safe.com/support/report-a-problem/ Thanks!

Thanks @danatsafe. Submitting a case now.

ebygomm
Influencer
Forum|alt.badge.img+31
  • ebygommInfluencer
  • Influencer
  • October 7, 2019

There used to be an issue with contains regex in the attributevalidator, although i thought it got fixed. Do any of the other attributevalidators use contains regex? If you remove that validation, do all features pass through?

https://knowledge.safe.com/questions/39814/attributevalidator-unexpected-results.html

mmccart
Contributor
Forum|alt.badge.img+16
  • mmccartContributor
  • Author
  • Contributor
  • October 7, 2019
ebygomm wrote:

There used to be an issue with contains regex in the attributevalidator, although i thought it got fixed. Do any of the other attributevalidators use contains regex? If you remove that validation, do all features pass through?

https://knowledge.safe.com/questions/39814/attributevalidator-unexpected-results.html

@egomm, I removed the REGEX validation statement and it passed all the records thru. What's odd is that in the AttributeValidators that are upstream, they contain REGEX statements and it works just fine. Very odd indeed.

mmccart
Contributor
Forum|alt.badge.img+16
  • mmccartContributor
  • Author
  • Contributor
  • October 7, 2019

While Safe looks into this issue, I replaced all of the AttributeValidator transformers with Tester transformers and the workbench runs as expected. Bonus: This change cut 38 seconds off the workbench run-time!

fmelizard
Contributor
Forum|alt.badge.img+17
  • fmelizardContributor
  • Contributor
  • October 21, 2019
mmccart wrote:

Thanks @danatsafe. Submitting a case now.

This problem with the AttributeValidator has been fixed in the latest 2019.2 and 2020.0 betas - https://www.safe.com/support/downloads/#beta

mark2atsafe
Safer
Forum|alt.badge.img+43
  • mark2atsafeSafer
  • Safer
  • Best Answer
  • October 21, 2019

The good news is the original issue is fixed. It's gone into various versions, so I'll list them all:

  • 2019.1: Build 19643
  • 2019.2: Build 19785
  • 2020.0: Build 20105

So check the version/build of the FME you are using and upgrade as necessary. 2019.2 is still in beta but I suspect just 2-3 weeks til it's released.

The not-so-good news is that I'd have to agree that the Tester is quicker at the moment. It's been updated to use "bulk mode" but the AttributeValidator has not. In time, that too will acquire bulk mode and be just about as quick as the Tester, but I don't know when that might be.

Reply


Helpful Members This Week

Community Stats

30,960
Posts
117,345
Replies
38,748
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.