Solved

Automatically remove unused attributes

  • 13 April 2023
  • 3 replies
  • 116 views
pflegpet
Rank
Badge +7

According to several solved questions in this forum, there are two recommended ways to automatically remove attributes that only conatain NULL or empty values: using a small python script or using the NullAttributeMapper. Unfortunately both suggestions do not work for me (FME Desktop/Form 2022.2). Both the PythonCaller and the NullAttributeMapper keep the Null-Attributes and they are present when I write to Excel (see attached workspace). Is there something wrong in my workflow or is there an updated solution? Thanks for any suggestions!

icon

Best answer by pflegpet 14 April 2023, 13:52

View original

3 replies

hkingsbury
Rank
Userlevel 5
Badge +29

As you're looking to dynamically change the schema (ie the schema is dependent/defined by the data) your going to need to follow a dynamic workflow where you make use of schema attributes and dynamic writers

https://community.safe.com/s/article/dynamic-workflow-tutorial-introduction

 

To achieve what your looking for, i've made a minor addition to the Python in the SchemaSetter custom transformer - https://hub.safe.com/publishers/safe-lab/transformers/schemasetter

 

In the uploaded workbench, you can see the changes i've made on lines 84-86. All my change does is that if the attribute does not have a value, it will exclude it from the schema creation. It then flows into a dynamic writer that uses the schema list on the attribute to define the excel sheets output fields

 

 

 

Check out the Abley FME Firesides on Youtube: https://www.youtube.com/@ableyltd
pflegpet
Rank
Badge +7

Hi @hkingsbury​ , thank you! Your modified Python code now removes all unused attributes. I also found out, that you could also use the SchemaScanner to achieve the same result (also using a dynamic writer). You just have to set it to "Ignore empty attributes" and exclude the schema attributes starting with fme_ (which is basically what you did in your script). I attached both solutions in case anyone needs this as a reference in the future.

hkingsbury
Rank
Userlevel 5
Badge +29

Hi @hkingsbury​ , thank you! Your modified Python code now removes all unused attributes. I also found out, that you could also use the SchemaScanner to achieve the same result (also using a dynamic writer). You just have to set it to "Ignore empty attributes" and exclude the schema attributes starting with fme_ (which is basically what you did in your script). I attached both solutions in case anyone needs this as a reference in the future.

Good to know re the SchemaScanner, I don't deal with dynamic processes all that much and haven't had the chance to use that yet, but will investigate it further :)

Check out the Abley FME Firesides on Youtube: https://www.youtube.com/@ableyltd

Reply


Community Stats

28,724
Posts
109,519
Replies
36,564
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.