Skip to main content

I am trying to compare the total of features from my original data source to the number of features  ultimately written to a dataset.   I thought I’d create a variable from Statistics Calculator result to refer to in my Tester.    Is there a way to reference the global variable I have set within the Tester phrase ?

If not, is there an efficient way of comparing the stats between the beginning and the end ?

 

You can use a FeatureMerger to merge both streams before using a Tester or TestFilter.


If I’m understanding the question correctly, you could simply move the VariableRetriever to the output port of the FeatureWriter, before the tester. You can then compare _total_features_written to the retrieved variable value.


I virtually never use VariableSetter or VariableRetriever these days.  There is almost never a use case for them since Safe have upgraded FME and brought in new functionality.  They are very hard to use and don’t support partial runs very well, which means I virtually always instead set a different transformer as something to “hold” the variable value (like a StatisticsCalculator etc. below) and I “retrieve” this by merging it back in with a FeatureJoiner.

Can compare Count Features Read with Count Features to be Written just by comparing the outputs of 2 StatisticsCalculator(s) and using FeatureJoiner

 




A variation to this to look at the Count of Features reported as Written by the FeatureWriter is similar, this time again with FeatureJoiner to compare the cached value of count of Features Read against the number reported as Written.

 


 

 


...otherwise if want to continue with VariableSetter/VariableRetriever, then general design I use is something like this.  This absolutely makes sure that

  • VariableSetter will always get called before VariableRetriever by sending the “main” data flow and the VariableSetter data flow into a Blocker Transformer
  • Making sure that VariableSetter outputs zero features and zero attributes downstream (They are not needed and will only confuse downstream transformers as to the “real” workspace Schema)

 

However, because can see there are actually more Transformers needed and I have to do something like put VariableSetter and VariableRetriever into a common collapsed Bookmark (to make sure they absolutely will fire together in Partial Runs, otherwise you can get into situation that VariableRetriever will try to run in a Partial Run when VariableSetter does not run and VariableRetriever will give the wrong output)……….then that’s why I stopped using them, and rely pretty much exclusively on FeatureJoiner as the Variable “Retriever”.

 


Wow…..so many ways to go about it.

I have dropped the use of VariableSetter  all together and taken these steps on board.

Thanks @bwn for such a thorough response.

 


Reply