Skip to main content

Hello there,

I would like to check a dataset attributes. So I would like to write a workspace that can check the incoming dataset.

For example, I expect to receive a dataset that has attribute name.

 

My expectation dataset attribute name: A, B & C

The incoming dataset attribute name: B, C & D.

 

In this case, I would like to call out attribute A and D. As attribute A is what I expected but missing and D is not the attribute name that I want.

 

May I ask what transformer I should use in this case?

 

Thank you very much.

 

Zachary

I think you have multiple options:

  • Schema Feature. Use the FeatureReader to read only the Schema Feature. Attributes to Features using the ListExploder.
  • AttributeExploder. Use the FeatureReader to read one feature. Attributes to Features using the AttributeExploder. (This will be performance intensive when processing a lot of features with a lot of attributes so probably less optimal.)

Compare features against a lookup table with the attributes you expect and act as you need.


I think you have multiple options:

  • Schema Feature. Use the FeatureReader to read only the Schema Feature. Attributes to Features using the ListExploder.
  • AttributeExploder. Use the FeatureReader to read one feature. Attributes to Features using the AttributeExploder. (This will be performance intensive when processing a lot of features with a lot of attributes so probably less optimal.)

Compare features against a lookup table with the attributes you expect and act as you need.

Thank you for your help. However, I still not sure if I know how to set up the workspace. I have tried and it couldn't work. Could you please explain a little bit more?

 

Sorry for the trouble and thank you for your help.

 

Have a good weekend!

 

Zachary


Thank you for your help. However, I still not sure if I know how to set up the workspace. I have tried and it couldn't work. Could you please explain a little bit more?

 

Sorry for the trouble and thank you for your help.

 

Have a good weekend!

 

Zachary

I think you want to do something like this:

2022-05-02_07h15_23


Thank you for your help. However, I still not sure if I know how to set up the workspace. I have tried and it couldn't work. Could you please explain a little bit more?

 

Sorry for the trouble and thank you for your help.

 

Have a good weekend!

 

Zachary

Thank you so much for your help! However, I do have a few questions would like to clarify.

1) the purpose of the 1st FeatureReader and AttributeExposer are to scan all the input files in a single folder?

2) the purpose of the 2nd FeatureReader is to read all the input files and explode all the attributes' name by using ListExploder?

 

Thank you and sorry for the troubles.

 

Zachary


Thank you for your help. However, I still not sure if I know how to set up the workspace. I have tried and it couldn't work. Could you please explain a little bit more?

 

Sorry for the trouble and thank you for your help.

 

Have a good weekend!

 

Zachary

1) the purpose of the 1st FeatureReader and AttributeExposer are to scan all the input files in a single folder?

Yes.

2) the purpose of the 2nd FeatureReader is to read all the input files and explode all the attributes' name by using ListExploder?

This reads only the schema of the file found by the first FeatureReader. The Schema feature contains a list with all column names. The ListExploder converts the listelements to features.


Thank you for your help. However, I still not sure if I know how to set up the workspace. I have tried and it couldn't work. Could you please explain a little bit more?

 

Sorry for the trouble and thank you for your help.

 

Have a good weekend!

 

Zachary

Got it. Thank you so much for your help. I am really appreciate that! :)

 

Zachary


Reply