Question

Workaround to run LineCombiner in Group mode? (OR condition)

  • 8 December 2022
  • 5 replies
  • 3 views

Badge +14
  • Contributor
  • 120 replies

I would like to combine lines on the attributes in column A OR B OR C and so on. The "combine on attributes" is an AND condition. What would be the cleanest workaround to achieve this?


5 replies

Userlevel 3
Badge +26

Would 3 LineCombiners in a row give the expected results, one for each attribute?

Badge +14

Would 3 LineCombiners in a row give the expected results, one for each attribute?

It would absolutely do the job. Only thing is that I would need +10 of them. Was hoping for some clever trick to cut the number of transformers.

Userlevel 5
Badge +25

Can you elaborate a bit more on your data structure? Are columns A, B and C always present and do they always have a value? Can they have the same value?

Badge +14

Can you elaborate a bit more on your data structure? Are columns A, B and C always present and do they always have a value? Can they have the same value?

The columns are always present, they do not always have a value but they can have the same value.

 

The LineCombiner in question is embeded in a custom transformer that is used in multiple streams. My plan B is to filter out only the columns that are needed for each respective stream. Before I venture into that I wanted see if there was some less convoluted way to mangage the group processing.

Badge +2

@aron​ Can you create a new group attribute using a conditional new_attr A if A NOT <null> or B if B NOT <null> OR C if C NOT <null and then use that?

Reply