Skip to main content
Question

Identify Columns with All Null values / AttributeValidator?

  • December 11, 2024
  • 4 replies
  • 83 views

galigis
Enthusiast
Forum|alt.badge.img+20

Hi All,

I have a feature class with 351 columns and need to know which columns contains ALL the records as <Null>. I have tried the AttributeValidator transformer but it doesn’t seem to do what I want as all the records go to Failed and there’s definetly columns with all records as <Null>, see below:

 

The ‘Attributes to Validate’ have been included the 351 columns,

Any ideas how to approach this issue?

Thanks :)

4 replies

liamfez
Influencer
Forum|alt.badge.img+35
  • Influencer
  • December 11, 2024

My immediate thought would be to put these all in a list that way you have one feature with lists for the attributes. Then you can remove duplicates from the lists and test to see which lists then only have 1 element (and that the element is null). This is pretty clunky, especially if you have a lot of columns, so I am hoping there is a better way and will keep thinking about it.


galigis
Enthusiast
Forum|alt.badge.img+20
  • Author
  • Enthusiast
  • December 11, 2024

Thanks ​@liamfez I was able to identify those fields with all <null> values by following the steps explained by ​@nielsgerrits here: 

 

...the total number of attribute fields with all <null> are 31.

 

How can I drop those fields from the original data? I’m thinking about using some dynamic gdb writer, see below:

 


liamfez
Influencer
Forum|alt.badge.img+35
  • Influencer
  • December 12, 2024

That looks good.

Yes I think a dynamic writer would be the way assuming no prior knowledge of which columns are entirely null. You could potentially use your attribute selection to make a schema and then have your original data align to that.


galigis
Enthusiast
Forum|alt.badge.img+20
  • Author
  • Enthusiast
  • December 12, 2024

Thanks ​@liamfez for the feedback...yes I’m struggling to transpose the rows into columns and having the schema before writing the data into the gdb…

...if you know any trick to transpose the following table into columns please let me know! :)

 

 


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings