Skip to main content

How can I control the flow of features when I have multiple flows of data in one workspace?

The Sampler coupled with a blocking transformer can be used to control the flow of features inthe workspace, when there are multiple flows of data in one workspace.

Sampler and blocking transformer control flow of features

A Sampler, with Sampling Amount set to 0, canbe used to control the order of data processing in the workspace. After the Sampler, place a blockingtransformer such as the FeatureHolder, FeatureMerger, or any other blocking ansformerwhich waits until all features have been received before processing. Route features from before the Sampler (orother features in the workspace) and Sampler output to the blocking transformer. TheSampler creates dependency between transformers. The blocking transformer will not finish processinguntil all features from the other branch of the workspace have finishedprocessing.

Sample workspace: controlflowusingthesampler.fmwt


The Sampler coupled with a blocking transformer can be used to control the flow of features inthe workspace, when there are multiple flows of data in one workspace.

Sampler and blocking transformer control flow of features

A Sampler, with Sampling Amount set to 0, canbe used to control the order of data processing in the workspace. After the Sampler, place a blockingtransformer such as the FeatureHolder, FeatureMerger, or any other blocking ansformerwhich waits until all features have been received before processing. Route features from before the Sampler (orother features in the workspace) and Sampler output to the blocking transformer. TheSampler creates dependency between transformers. The blocking transformer will not finish processinguntil all features from the other branch of the workspace have finishedprocessing.

Sample workspace: controlflowusingthesampler.fmwt

This is a nice workaround, but is it planned to have a "wait" transformer to which we could provide a transformer name to wait for (to simply do this whitout the memory usage of the feature holder ?)

 

 


The Sampler coupled with a blocking transformer can be used to control the flow of features inthe workspace, when there are multiple flows of data in one workspace.

Sampler and blocking transformer control flow of features

A Sampler, with Sampling Amount set to 0, canbe used to control the order of data processing in the workspace. After the Sampler, place a blockingtransformer such as the FeatureHolder, FeatureMerger, or any other blocking ansformerwhich waits until all features have been received before processing. Route features from before the Sampler (orother features in the workspace) and Sampler output to the blocking transformer. TheSampler creates dependency between transformers. The blocking transformer will not finish processinguntil all features from the other branch of the workspace have finishedprocessing.

Sample workspace: controlflowusingthesampler.fmwt

NatalieAtSafe answer it's a very good answer if you want to keep the features isolated.

 

 

Maybe this can be useful. You can do several transformations over features in parallel and join the results with an Aggregator transformer. In case you have more than one original features, you can use the "Group by" parameter.

 


I've often thought it would be useful to have the transformer equivalent of an electronic transistor. Use one set of features to control another set. Or the end of a stream to trigger the start of another stream.


I've often thought it would be useful to have the transformer equivalent of an electronic transistor. Use one set of features to control another set. Or the end of a stream to trigger the start of another stream.

Hi @russell, probably the FeatureMerger can be used to control feature flow as your expected. In the workflow shown as below, the Unusedsupplier port starts outputting the Supplier features after the UnmergedRequestor port has finished outputting all the Requestor features.

 


Reply