Skip to main content

Since last couple of versions this really annoying message pop's up.

As soon as one or more merge keys has duplicates. Even if the combined keys are unique.

I am forced to put a (disabled) inspector on the rejected features output port.

PCNLTSTRAATNAAM2324HG702Bachstraat2324HG704Bachstraat2324HG706Bachstraat

 

In the example above I use both fields as merge keys. The "straatname" causes the extra_ref._feature rejection.

This gets worse if I merge on more fields, which I do in this workbench the example is from, up to 7 fields. (setting ignore nulls or empty will cause everything to not merge, as all of them have 1 or more with value null)

There is no setting to "ignore rejection" afaik.

Of course I can concatenate first into a merge_key to emulate oracle's "coalesce". But shouldn't be necessary I think.

Anyone knows why? Or is there a setting to stop this behavior.

Is there a "ignore rejection" option?

Hi @gio, try checking the Process Duplicate Suppliers.


@takashi

Hi Takashi,

I don't need to process duplicates. And it therefore should not bug me with the "extra_reference_feature".

The point is that there are no duplicates technically, only some of the merge keys are, and some are even null or empty so technically duplicate key.

7keys.png

Here are 7 keys. The keys combined are unique. Only separate keys might and are none-unique.

There is no need to waste time concatenating the keys in to a single key here as I am supposed to be able to merge on multiple keys.

But the transformer seems to judge individual keys and pass them to the rejected port, forcing me to add a disabled inspector to this port.

I understand the logic of the merging. I do state that 1 or more single keys out of multiple keys being non-unique should not yield a duplicate detection if the total keys are unique.

And even so, it should not halt the process (when no inspector is connected).


@gio,

What about changing the workspace parameters on how the rejected features are handled?

It will still say that you have extra reference features, but it wont terminate your translation and you wont need to add a disabled inspector.


So there is a setting to have it not terminate translation.

But this does not address my complaint.

There is NO extra reference feature.

The combined merge-keys are unique.

The transformer starts nagging as soon as there is one or more sub-keys of the key-complex that is not unique!

It therefore should not report extra reference keys. No object should be rejected, they should be passed to the unreferenced port.


So there is a setting to have it not terminate translation.

But this does not address my complaint.

There is NO extra reference feature.

The combined merge-keys are unique.

The transformer starts nagging as soon as there is one or more sub-keys of the key-complex that is not unique!

It therefore should not report extra reference keys. No object should be rejected, they should be passed to the unreferenced port.

Hi @gio, can you share a workspace with a small sample of your data that could demonstrate the issue? It might help us better understand what's happening.

So there is a setting to have it not terminate translation.

But this does not address my complaint.

There is NO extra reference feature.

The combined merge-keys are unique.

The transformer starts nagging as soon as there is one or more sub-keys of the key-complex that is not unique!

It therefore should not report extra reference keys. No object should be rejected, they should be passed to the unreferenced port.

This then sounds like a full on bug. Can you send a repro over to support@safe.com to be sure we can duplicate and understand (and fix?)

 

 


This then sounds like a full on bug. Can you send a repro over to support@safe.com to be sure we can duplicate and understand (and fix?)

 

 

More -- I just tried to recreate the same situation but failed to produce any odd behaviour. Definitely need your repro sample.