Bulk Mode in PythonCaller

Thanks Mark :)

For future reference, I’ll note that “preserve” is the important part, here. Setting the has_support_for flag for FME_SUPPORT_FEATURE_TABLE_SHIM has the effect of splitting a feature table into individual features that are tagged as belonging to a bulk mode feature table, and will be reassembled at output.

The C/C++ SDK describes it like this (from fmeobjects\cpp\fmesupporttype.h):

enum SupportType
{
// ...
   // Whether a plugin supports feature tables via a "shim", which passes the feature table to the
   // plugin as if it was features (i.e. instead of getting one feature with an IFMEFeatureTable
   // geometry, the plugin will get N features). This can allow plugins to preserve feature tables
   // without explicitly handling them, which may improve performance. Note that some calls on the
   // IFMEFeature interface are not supported for shim features, and calling them will cause the
   // feature table to be split into individual features, potentially losing the performance
   // benefit.
   // Factories that opt in for shim support are required to either output or destroy all input
   // features during the process() call (i.e. the factory cannot store a feature and use it later).
   FME_SUPPORT_FEATURE_TABLE_SHIM = 1,

// ...

};

The fmeobjects Python library doesn’t let you create bulk mode features like this, which would make this feature only useful for attribute manipulation on existing features (something that the PythonFactory doc calls “Single Input-Single Output style”). However, there is a workaround abusing the “preserve” mechanism that lets you pass Python-created features to a factory pipeline that would output them in bulk mode, and then returning the output of that pipeline as the output of your PythonCaller. Since these features are now tagged as being part of a Bulk Mode feature table, they get reassembled by FME before it sends them downstream, something that can tremendously cut down on time spent waiting for the next transformer to accept/finish processing the features your PythonCaller outputs.