Question

How to manage input order ?

  • 3 April 2014
  • 2 replies
  • 3 views

Badge
I've got a workbench which look like this:

 

 

  Reader --> T1 ---------->Aggregator -> Writer

 

                 |              |

 

                 -> CT1 -|

 

                 -> CT2 -|

 

 

Every feature has an "id" and the aggregator groupd them with the "id"

 

 

T1 is a simple transformer (like AttributeCreator).

 

CT1 and CT2 are custom transformer very complicated with many steps (~20 each without the same length).

 

 

Because i've got ~1million element in the Reader, in the Aggregator I which I could select the "Input is ordered by group" for "performance" issues.

 

But it does not work; I've got only the T1 output.

 

 

So, How may I do this ?

 

 

For feeding the pipe I though of "no-op" block like in electronic ... but it's does not exist and it's a pain to do.. and not very reliable.

 

 

Is there something else I could do / use ?

 

 

 

 

 

2 replies

Badge +3
Hi,

 

 

If every item has an (unique)ID then grouping on that ID is not very usefull for perfomance gain.

 

 

If u want the gain and the ability to use paralel processing you should create some grouping strategy.

 

U could use a gridbuilder, or a tiler strait at the beginning of the bench. I usualy use ie neighborhoodnames or create some grid. Teh attributes must be passed all trough the processing. Alos take care to keep the correct instances of these "group_D's"

 

 

This also fascilitates paralell processing.

 

Greets

 

Gio
Badge
You're right: it's not very useful for performance, but it's mendatory for database insertion.

Reply