I have a need to sample 10% of a data set. I use the Counter and then the Statistics Calculator to get the total record count. From there, the Expression Evaluator is used to multiply that total count by 10% (setting to a whole number). The Variable Setter then sets that value to a variable. The variable is used for the N input on the Sampler. I do not get an error, but no records are output from Sampler. I have confirmed there is a value in that variable (using eh Variable Retriever). Enclosed are the details. Any suggestions?
Solved
Use Variable on Sampler
Best answer by takashi
Hi @chris_wright, if the sampling rate is always 10% (i.e. extract one-tenth of the features), the Sampler with these parameters does that.
- Sampling Rate (N): 10
- Sampling Type: Every Nth Feature
- Randomize Sampling: <set Yes if you need to perform random sampling>
+19- Safer
Hi @chris_wright,
I'm sorry, but the variable value can only be accessed by the VariableRetriever transformer, and is not available as an parameter input on any other transformer.
If you need to get the first 10% of the features, you can use a Tester to check whether the count attribute is less than (total count*0.1). This will run just as fast as using a Sampler.
- Best Answer
Hi @chris_wright, if the sampling rate is always 10% (i.e. extract one-tenth of the features), the Sampler with these parameters does that.
- Sampling Rate (N): 10
- Sampling Type: Every Nth Feature
- Randomize Sampling: <set Yes if you need to perform random sampling>
daveatsafe wrote:
Hi @chris_wright,
I'm sorry, but the variable value can only be accessed by the VariableRetriever transformer, and is not available as an parameter input on any other transformer.
If you need to get the first 10% of the features, you can use a Tester to check whether the count attribute is less than (total count*0.1). This will run just as fast as using a Sampler.
Reply
Rich Text Editor, editor1
Editor toolbars
Press ALT 0 for help
Helpful Members This Week
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.