Skip to main content
Question

Testing null values of several attributes in one shot

  • December 3, 2021
  • 7 replies
  • 541 views

user01
Contributor
Forum|alt.badge.img+2

Hello,

I want to know if 90 columns have NULL values.

How can I use Tester Transformer without testing each attribute one by one.

Can I use the Text Editor and write it directly and how ?

Thank you !

7 replies

nielsgerrits
VIP
Forum|alt.badge.img+60
  • 2938 replies
  • December 3, 2021

Yes you can. Fold out "Composite Expression" and click "Edit".

TesterEdit(2021)


ebygomm
Influencer
Forum|alt.badge.img+44
  • Influencer
  • 3427 replies
  • December 3, 2021

You can use the AttributeValidator to test for not null, much friendlier to use if you have 90 attributes

image 

 


Forum|alt.badge.img+2
  • 1891 replies
  • December 3, 2021

@user01​ All the suggested ideas are correct, but probably tedious if you have 90 attributes to test. I've created a python script that will test for nulls:

import fme
import fmeobjects
import re
 
# find the list of <null> attributes on a feature
 
def processFeature(feature):
    # initialize the python lists with FME lists
    _attrList = feature.getAllAttributeNames()
    #print(_attrList)
    _nullAttrs = []
    for i in range(len(_attrList)):
        attrValue = feature.getAttribute(_attrList[i])
        # filter out format and other unwanted attributes with regex
        x = re.search("^multi_|^csv_", _attrList[i])
        # test if attrValue is null and not in the unwanted list of attrs.
        if not attrValue and not x:
            _nullAttrs.append(_attrList[i])
    # output the FME lists
    feature.setAttribute('nullAttributes{}', _nullAttrs)

Workspace attached (FME 2021.2)


Forum|alt.badge.img+2
  • 1891 replies
  • December 3, 2021

@user01​ All the suggested ideas are correct, but probably tedious if you have 90 attributes to test. I've created a python script that will test for nulls:

import fme
import fmeobjects
import re
 
# find the list of <null> attributes on a feature
 
def processFeature(feature):
    # initialize the python lists with FME lists
    _attrList = feature.getAllAttributeNames()
    #print(_attrList)
    _nullAttrs = []
    for i in range(len(_attrList)):
        attrValue = feature.getAttribute(_attrList[i])
        # filter out format and other unwanted attributes with regex
        x = re.search("^multi_|^csv_", _attrList[i])
        # test if attrValue is null and not in the unwanted list of attrs.
        if not attrValue and not x:
            _nullAttrs.append(_attrList[i])
    # output the FME lists
    feature.setAttribute('nullAttributes{}', _nullAttrs)

Workspace attached (FME 2021.2)

including the sample data...


user01
Contributor
Forum|alt.badge.img+2
  • Author
  • Contributor
  • 9 replies
  • December 6, 2021

including the sample data...

Wouahou ! Tested on your sample data, great ! Thank you.

My data comes from 300 feature types in 1 gml file converted in ffs file. How can I adapt it ?


Forum|alt.badge.img+2
  • 1891 replies
  • December 6, 2021

including the sample data...

@user01​ Perhaps contact your local reseller to that this further with a more complex scenario


user01
Contributor
Forum|alt.badge.img+2
  • Author
  • Contributor
  • 9 replies
  • December 7, 2021

including the sample data...

I will ! Thank you for your help ! I'll post the solution when I'll find it.