Attribute > 0 appears to be consistent (the non-null values are all integers) so I will use that. I'm still concerned by the inconsistency of null - don't tell me - fixed in FME 2017! ;-)
I don't know why the Tester would be behaving like this, perhaps you could try using a TestFilter instead?
You could also try removing the Junction and routing the readers directly to the Tester. The Junction should not be affecting anything but it's worth ruling out all possibilities.
Also, to try to determine what the problem is, you could try placing a DuplicateFilter after the tester, to filter out distinct values, then a Logger to see what values actually exist after the Tester. As mentioned in the other article, the Inspector can sometimes show things not exactly as they should appear. The Logger might be a better choice. Of course, that all depends on if the DuplicateRemover is able to determine differences in the values. If DuplicateRemover doesn't work try using a StatisticsCalculator, grouping by your "null" attribute value.
Hi @tim_wood, I'm afraid that it could be a potential bug in a particular version of FME. Which version (build number) of FME are you using? Does the same symptom occur in the latest FME 2017.0 build 17271?
Hi @tim_wood
have you tried Attribute Is Null operator in Tester/TestFilter?
Thanks for the suggestions. I'm using FME 2016.1 (Build 16492). Has FME 2017 officially been released yet?
@LenaAtSafe - Attribute Is Null in the Tester was what I did first and what seemed to be giving variable results.
@nic_ran - I tried connecting the Readers directly to the Tester (Attribute Is Null) then each output port of the Tester to a separate Inspector. I ran this twice. Although the same number of records came out of each Tester output port (38717 allegedly not null), 1 record with a null value still ended up in the wrong place each time. But it wasn't the same record!
However, I purged the temporary files and ran it again and this time it matched the results for Attribute > 0 (38716 not null). So I had a look in the FME_TEMP folder and cleared out some other old stuff that was there. Then I re-ran the Workspace and got the correct results. I re-ran it again and this time I got 38720 records supposedly not null but the first 4 in the Inspector were null :-(
I noticed that whereas on the first run, no records went along the not null path for ages (until after over 100,000 records), the second time I got several records going down that path more or less straight away (4 I think but I can't be sure).
Could it be something to do with leaving the Inspector open when re-running the Workspace?
But I did this on the third run and this time, no records went down the not null path and I got the correct results.
For info, the bulk of the records are in the first FGDB to be read (289,000).
TestFilter report to follow when I get a chance.
How about trying an "attribute has value" test instead?
Or the NullAttributeMapper and test for the new value set for multiple conditions ( empty, null, missing, etc)?
Yes FME 2017 is ofiicially released....
Inspector open or not doesn't seem to make a difference.
I think the multiple source FGDBs can also be discounted because I tried it with just the first one and still had issues.
I purged temporary files then tried Attribute has a value. That seems to be more reliable based on the repeat runs so far but I think Attribute is Null should be just as reliable.
I will download 2017 and try that.
fme2016.1.1.0
I had Samish issue where suddenly tester used in 2015 didn't work no more.
I went for testing
NULL,MISSING AND EMPYT. ...just to be sure..lol
First 2016 version seemed to be not able to make up its mind though and even this .
I had one workspace where this did not do anything until I removed the empty test (a.f.a.i. remember)
Lately It seems to be able to run on MISSING AND NULL.
Thanks for the suggestions. I'm using FME 2016.1 (Build 16492). Has FME 2017 officially been released yet?
@LenaAtSafe - Attribute Is Null in the Tester was what I did first and what seemed to be giving variable results.
@nic_ran - I tried connecting the Readers directly to the Tester (Attribute Is Null) then each output port of the Tester to a separate Inspector. I ran this twice. Although the same number of records came out of each Tester output port (38717 allegedly not null), 1 record with a null value still ended up in the wrong place each time. But it wasn't the same record!
However, I purged the temporary files and ran it again and this time it matched the results for Attribute > 0 (38716 not null). So I had a look in the FME_TEMP folder and cleared out some other old stuff that was there. Then I re-ran the Workspace and got the correct results. I re-ran it again and this time I got 38720 records supposedly not null but the first 4 in the Inspector were null :-(
I noticed that whereas on the first run, no records went along the not null path for ages (until after over 100,000 records), the second time I got several records going down that path more or less straight away (4 I think but I can't be sure).
Could it be something to do with leaving the Inspector open when re-running the Workspace?
But I did this on the third run and this time, no records went down the not null path and I got the correct results.
For info, the bulk of the records are in the first FGDB to be read (289,000).
TestFilter report to follow when I get a chance.
FME 2017.0 has already been released. Please download the install from
http://www.safe.com/support/support-resources/fme-downloads/
May I ask you to run your workspace with FME 2017.0 to confirm whether there is still a problem with Tester testing differently/incorrectly? If FME 2017.0 has the same incorrect (hard to get correct) behaviour as FME 2016, could you please submit a support case? Please add link to this discussion to your problem description (no need to describe everything once again when submitting the case). Ideally, we would need you data - or a data sample that reproduces the wrong behaviour. We definitely want to investigate this problem. Sorry for the inconvenience caused by it - I am glad you caught the wrong output.