Skip to main content

Hello everyone,

 

This might be a simple question, but for some reason I can't figure out how to remove rows with only Null Values (Of course I can useTestFiler or Tester, but would be cumbersome work, since I have a lot of columns and I don't want to go into that rabbit hole)

These rows with Null values came into existence, since I read in an excel file with two worksheets with two different schemas. So the column names in worksheet 1, is different compared to the column names in worksheet 2.

Therefore all the values from worksheet 2 have null values. I actually don't need worksheet 2, so I was thinking there might be 2 solutions.

 

1) don't read in worksheet 2 or

2) remove the null values

 

For both of them I don't know the answer.....dramatic sound effects..

So if someone knows the answer, that would be very nice.

If I understand correctly you want to do this dynamic? So input to be defined? One way to do this is to use an AttributeExploder. This will return a feature for each cell with _attr_name and _attr_value. Then you can test for all records where _attr_value has no value. Then merge the results with an Aggregator and merge that back to the original row to shift features with values from features with no values. Be aware that this is memory intensive when having a lot of features and a lot of attributes.


Expanding on @nielsgerrits​ response, in the AttributeExploder you can use the Exploding Type of List which will put your attribute names and values into a consolidate list on the original features. You will need to use the Ignore Attributes Containing parameter to ignore fme format attributes, and any other attributes you aren't testing for (this is a regex input). After the AttributeExploder, you can use a ListConcatentor (making the Separator Character parameter blank). If your test attributes are null, the concatenated attributed from the ListConcatentor will be blank, which can easily be tested for in a Tester or TestFilter.


Expanding on @nielsgerrits​ response, in the AttributeExploder you can use the Exploding Type of List which will put your attribute names and values into a consolidate list on the original features. You will need to use the Ignore Attributes Containing parameter to ignore fme format attributes, and any other attributes you aren't testing for (this is a regex input). After the AttributeExploder, you can use a ListConcatentor (making the Separator Character parameter blank). If your test attributes are null, the concatenated attributed from the ListConcatentor will be blank, which can easily be tested for in a Tester or TestFilter.

Yes, using the list is a far better solution. ListElementCounter instead of ListConcatenator might be quicker an will work as well.


@nielsgerrits​ @dustin​ I feel bad, since I found another solution so only features are read from the first worksheet from excel. I found it based on the following

post.

Using two FeatureReaders and one Sampler in between. I used the followings settings for the sampler and the 2nd FeatureReader with a Constraint based on Features Type to Read: fme_feature_type_name. Now only feature from the first worksheet are read. Since @dustin​ answer is actually more the answer to my question (how to remove Null values), I will select that post as "Select as Best"

afbeeldingafbeelding


Reply