Skip to main content
Question

PythonCaller Works Internally But Doesn't Pass Features


diannebcgray
Contributor
Forum|alt.badge.img+1
FME Workflow

Hi, I would like some help with PythonCaller.  I have a workflow where I read in feature classes from an Esri file geodatabase, combining the features into a single attribute to make it dynamic.  I filter out annotation, resolve domains and rename original domain fields, map empty and missing features to Null, then use AttributeFilter to split by FME Feature Type.

My next desired steps are to identify, for each FME Feature Type, which attributes are nulls, and remove those attributes, then connect all splits to a Writer to write out to shapefiles grouped by FME Feature Type.

I am trying to use PythonCaller to remove the null attributes.  It stores the attribute names in a list, and stores in a dictionary called alnum_tracker attribute names and whether they contain data (where they are saved as True), before comparing the list of attributes to the dictionary, deleting any that are not matches.  I’ve tested it and it is grabbing the correct attributes to delete and keep.  The Python works, and my logged messages output expected results from the data.  However, when I use pyoutput to output features, I get nothing.  No features get outputted.  I get the messages in my Translation Log: 

“Destination Feature Type Routing Correlator (RoutingFactory): Tested 77 input feature(s), wrote 77 output feature(s): 0 matched merge filters, 0 were routed to output, 77 could not be routed”

“Final Output Nuker (TeeFactory): Cloned 77 input feature(s) into 0 output feature(s)”.

I’m new to FME and using PythonCaller in FME.  I want to know why I can’t get output features from PythonCaller.  My workflow is shown above, and my code below.

import re
import fmeobjects

class FeatureProcessor:
	def __init__(self):
		self.alnum_tracker = {}
		self.feature_list = []
		self.pattern = re.compile(r'[a-zA-Z0-9]')

	def input(self, feature):
		self.feature_list.append(feature)

		for attr in feature.getAllAttributeNames():
			val = feature.getAttribute(attr)

			if isinstance(val, str) and self.pattern.search(val):
				self.alnum_tracker[attr] = True
			elif isinstance(val, (int, float)) and val is not None:
				self.alnum_tracker[attr] = True

	def close(self):
		fmeobjects.FMELogFile().logMessageString(f"Number of features collected: {len(self.feature_list)}", fmeobjects.FME_INFORM)
		fmeobjects.FMELogFile().logMessageString("Alnum tracker: " + str(self.alnum_tracker), fmeobjects.FME_INFORM)

		if not self.feature_list:
			return

		all_attrs = self.feature_list[0].getAllAttributeNames()
		attrs_to_remove = [attr for attr in all_attrs if attr not in self.alnum_tracker]

		fmeobjects.FMELogFile().logMessageString("Attributes to remove: " + str(attrs_to_remove), fmeobjects.FME_INFORM)

		for feat in self.feature_list:
			for attr in attrs_to_remove:
				feat.removeAttribute(attr)
			fmeobjects.FMELogFile().logMessageString("Outputting feature", fmeobjects.FME_INFORM)
			fmeobjects.FMELogFile().logMessageString("Feature type: " + feat.getFeatureType(), fmeobjects.FME_INFORM)
			original_type = feat.getFeatureType()
			feat.setFeatureType(original_type)
			fmeobjects.FMELogFile().logMessageString(f"Feature list length: {len(self.feature_list)}", fmeobjects.FME_INFORM)
			protected_attrs = {"_feature_type", "fme_feature_type", "routing_key"}
			attrs_to_remove = [attr for attr in all_attrs if attr not in self.alnum_tracker and attr not in protected_attrs]
			self.pyoutput(feat)
			fmeobjects.FMELogFile().logMessageString("Remaining attributes: " + str(feat.getAllAttributeNames()), fmeobjects.FME_INFORM)

How do I make my Python output talk to my data properly so that I get output features from it that are missing the null attributes?

 

6 replies

takashi
Evangelist
  • June 19, 2025

Hi ​@diannebcgray ,

In your screenshot, the Feature Counting indicates 77 features have been output from the PythonCaller. I think the “pyoutput” method in your script works fine.

 


david_r
Celebrity
  • June 19, 2025

If you need to specifically remove attributes with null values, you could do something like:

for attr in feature.getAllAttributeNames():
    if feature.isAttributeNull(attr):
        feature.removeAttribute(attr)

Or modify it to your use case, the key is to use the isAttributeNull() method to detect proper null.


diannebcgray
Contributor
Forum|alt.badge.img+1
  • Author
  • Contributor
  • June 19, 2025
takashi wrote:

Hi ​@diannebcgray ,

In your screenshot, the Feature Counting indicates 77 features have been output from the PythonCaller. I think the “pyoutput” method in your script works fine.

 

Yes, 77 features are passed through, with nothing changed and no null attributes removed.


takashi
Evangelist
  • June 20, 2025

I think your script works to remove every attribute that matches an element in the "attrs_to_remove" list.

How have you determined the attributes have not been removed?

Just be aware, the names of removed attributes could be still exposed as column header in the Table View of Visual Preview or FME Data Inspector, but they have been removed actually if "<missing>" were shown as their values. 


diannebcgray
Contributor
Forum|alt.badge.img+1
  • Author
  • Contributor
  • June 20, 2025
takashi wrote:

I think your script works to remove every attribute that matches an element in the "attrs_to_remove" list.

How have you determined the attributes have not been removed?

Just be aware, the names of removed attributes could be still exposed as column header in the Table View of Visual Preview or FME Data Inspector, but they have been removed actually if "<missing>" were shown as their values. 

I have used the Inspector transformer, but I have also checked it by writing out to a shapefile output.  The empty fields are still there when I want them gone.


takashi
Evangelist
  • June 21, 2025

When you check the result with Inspector, does <missing> appear in the fields like this?

In the FME Table View - Workbench Visual Preview or Data Inspector, <missing> indicates that the attribute does not exist - in other words, it has been removed. However, if you have defined the missing fields in a writer feature type as User Attributes, the writer would write out the attributes into the destination dataset as null.


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings