I have a need to sample 10% of a data set. I use the Counter and then the Statistics Calculator to get the total record count. From there, the Expression Evaluator is used to multiply that total count by 10% (setting to a whole number). The Variable Setter then sets that value to a variable. The variable is used for the N input on the Sampler. I do not get an error, but no records are output from Sampler. I have confirmed there is a value in that variable (using eh Variable Retriever). Enclosed are the details. Any suggestions?
Solved
Use Variable on Sampler
Best answer by takashi
Hi @chris_wright, if the sampling rate is always 10% (i.e. extract one-tenth of the features), the Sampler with these parameters does that.
- Sampling Rate (N): 10
- Sampling Type: Every Nth Feature
- Randomize Sampling: <set Yes if you need to perform random sampling>
This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
+19- Safer
- 1637 replies
Hi @chris_wright,
I'm sorry, but the variable value can only be accessed by the VariableRetriever transformer, and is not available as an parameter input on any other transformer.
If you need to get the first 10% of the features, you can use a Tester to check whether the count attribute is less than (total count*0.1). This will run just as fast as using a Sampler.
- 7843 replies
- Best Answer
Hi @chris_wright, if the sampling rate is always 10% (i.e. extract one-tenth of the features), the Sampler with these parameters does that.
- Sampling Rate (N): 10
- Sampling Type: Every Nth Feature
- Randomize Sampling: <set Yes if you need to perform random sampling>
Hi @chris_wright,
I'm sorry, but the variable value can only be accessed by the VariableRetriever transformer, and is not available as an parameter input on any other transformer.
If you need to get the first 10% of the features, you can use a Tester to check whether the count attribute is less than (total count*0.1). This will run just as fast as using a Sampler.