Relatively new FME 2018 user and just trying to see if there some improvements to my use of Tester in the workspace. Now, I've been using Tester a lot to do simple things like using it like a WHERE condition, but in 2018 at least it seems to do a naive, feature-by-feature test of the Tester conditions. For example if I use Tester on a 2M feature input with a condition AttributeValue = 'A', then this takes around 2 minutes to complete.
However, I've later discovered FeatureJoiner as a go-to transformer for significantly better performance as this seems to first index the Attribute(s) in question first before testing them, as well as passing out the Unjoined features without any post-processing overhead as essentially "untouched" features.
If I instead use an alternative of Creator(1 feature)->AttributeCreator(Set AttributeValue='A') to make a single Test Feature, and then pass this into the RHS of a FeatureJoiner transformer to test against the 2M feature set on the LHS, then the same filter/test then only takes 11 seconds to complete, because it appears FeatureJoiner first creates an index on the feature set to be tested/joined against vs Tester which doesn't.
So, am I using Tester wrong or applying it in the wrong context? I know it can do a lot of other test types but "=" type queries I would have thought would be its bread-and-butter? Is there any other way to wring out attribute indices in other Transformers like Tester, FeatureMerger etc. that similarly get outperformed by transformers that make use of indices like FeatureJoiner?
Best answer by mark2atsafe
View original