As I’m writing this FME 2024.0 just added support for multiple-port Python transformers this week (though that’s not supported on PythonCaller yet, users need to use the Transformer Designer and figure out how to use FME’s Tcl macros for that). That change has been a long time coming, but it is a massive step in improving the flexibility of Python-based transformers and is going to unlock some extremely useful use-cases.
However, I believe it is still lacking a few parts, namely a hook to signal to the Python transformer that an input port is closed and will no longer be recieving any features (a role usually filled by close() when there’s only one port) and bulk mode transport that can vary across ports. For context, consider all the flow control issues I’m pointing out in this other idea:
Making a rudimendary version of the sort of transformer I’m proposing in this idea isn’t too complicated and ends up looking like this:
The idea here is using a FeatureWriter as a proxy for knowing when the PriorityIn port closes, and not letting any features from DelayedIn through before then, but otherwise letting PriorityIn features flow freely, and ideally letting DelayedIn features flow just as freely once PriorityIn is closed (in practice, FeatureHolder actually blocks all DelayedIn features until the last one has arrived).
It wouldn’t be possible to make such a transformer with the 2024.0 Python changes alone, due to how close() is only called when all input ports have sent their “All done” signal, instead of having a separate close method that gets called for each port. Not being able to configure individual ports for bulk mode support is also going to be a problem, because DelayedIn features that come in early will probably need to be buffered until PriorityIn closes (which is illegal in bulk mode), but PriorityIn can still operate in bulk mode just fine, and in fact so can DelayedIn if PriorityIn is already closed by the time the first feature comes in.
That means that, in order to fulfill that use case, two new methods should be added alongside the new input_from():
input_port_has_support_for(self, support_type, input_tag), called once per port, but only before the first feature for that port comes in, so the behavior can change based on the state of other ports.has_support_for()would probably no longer need to be called if this one is present, based on my current understanding of bulk mode.input_port_closed(self, input_tag, is_connected), called when the upstream node(s) to a given port signals it has no more features to send, or some time between initialization and the first feature being sent to any port for ports that are not connected to anything.
I’m only using the priority fence example because it’s the easiest I have to explain that requires both, but these additions would be very useful for a number of cases that are currently tricky to handle, whether with Python or other transformers.
