Add Rejected-port to the PythonCaller

I like it. It might be ideal if the user could add (and rename) output ports optionally. How about this?

Even better :-)

Yes, that would be really great! Filtering on the spot. I now always tag my features with some attribute and use a TestFilter afterwards or something.

That is a great idea, I think 90% of the time my pythonCaller is immediately followed by a tester/TestFilter to separate the results.

How about an additional port for a failing Python script instead of making the whole workspace fail?

It would be great if this port included a Line Number, Type Name and settable Context attribute, e.g.

Why not go whole hog and have a Python-based transformer designer API?

self.pyoutput_failed(feature)

It can be very interesting for all users.

Seems like it has been added to FME 2024.0, according to fmetools.

I’m noticing a few things with the 2024.0 changes to support this in the underlying PythonFactory:

• The PythonCaller itself hasn’t yet been modified to allow multiple ports
• The default template of the Transformer Designer still uses an upstream TeeFactory and a downstream TestFactory to simulate multiple output ports instead of plugging all ports in the PythonFactory directly
• Calling self.pyoutput(feature, "<REJECTED>") will end the workflow as soon as the Rejected feature appears, like most other transformers, but a raised exception (not even an FMEException) will not be turned into a rejected feature. Errors still need to be properly caught and the feature created by the exception handler.
• The close() method only fires once and does not have a per-port identifier, meaning a Python script cannot validate that individual input ports have finished sending data. This means transformers that need to wait for all features to have come through to start outputting can’t do so independently between ports, and order-critical operations can’t be asserted.

 I made a 2024.0-only PythonCaller clone with a <Rejected> port just to try it out and show how it works. I can’t attach any files here, and this is by no means tested enough to be uploaded to FME Hub, so I’m just going to dump the contents of the FMXJ file here.

Documents\FME\Transformers\PythonCallerRejectable.fmxj

{
    "info": {
        "categories": [
            "Workflows"
        ],
        "comments": [
            ""
        ],
        "name": "PythonCallerRejectable"
    },
    "versions": [
        {
            "changeLog": [
                "Initial version of transformer"
            ],
            "comments": [
                "================================ Version 1 ========================================"
            ],
            "executionTemplate": [
                "",
                "FACTORY_DEF {*} PythonFactory",
                "   FACTORY_NAME { $(XFORMER_NAME) }",
                "   PY_INPUT_TAGS Input",
                "   $(INPUT_LINES)",
                "   SYMBOL_NAME { $(PYTHONSYMBOL) }",
                "   PYTHON_NAMESPACE FMEOBJECTS",
                "   SOURCE_CODE { $(PYTHONSOURCE) }",
                "    PY_OUTPUT_TAGS Output <REJECTED>",
                "    OUTPUT { Output FEATURE_TYPE $(OUTPUT_Output_FTYPE)",
                "        $(OUTPUT_Output_FUNCS) }",
                "    OUTPUT { <REJECTED> FEATURE_TYPE $(OUTPUT_<REJECTED>_FTYPE)",
                "        $(OUTPUT_<REJECTED>_FUNCS) }",
                ""
            ],
            "featureHolding": "none",
            "form": {
                "parameters": [
                    {
                        "name": "TRANSFORMER_GROUP",
                        "parameters": [
                            {
                                "name": "XFORMER_NAME",
                                "prompt": "Transformer Name",
                                "showEditButton": false,
                                "supportedValueTypes": [
                                    "none"
                                ],
                                "type": "text",
                                "valueType": "string"
                            }
                        ],
                        "prompt": "",
                        "type": "group",
                        "valueType": "string"
                    },
                    {
                        "defaultValue": "",
                        "name": "PARAMETERS_GROUP",
                        "parameters": [
                            {
                                "defaultValue": "FeatureProcessor",
                                "editor": "plaintext",
                                "name": "PYTHONSYMBOL",
                                "prompt": "Class to Process Features",
                                "required": true,
                                "showArithmeticEditor": false,
                                "showEditButton": false,
                                "showInlineEditor": false,
                                "showPrompt": true,
                                "supportedValueTypes": [
                                    "none"
                                ],
                                "trimWhitespace": true,
                                "type": "text",
                                "valueType": "string"
                            },
                            {
                                "defaultValue": "import<space>fme<lf>import<space>fmeobjects<lf><lf>class<space>FeatureProcessor<openparen>object<closeparen>:<lf><space><space><space><space><quote><quote><quote>Template<space>Class<space>Interface:<lf><space><space><space><space>When<space>using<space>this<space>class<comma><space>make<space>sure<space>its<space>name<space>is<space>set<space>as<space>the<space>value<space>of<space>the<space><apos>Class<space>to<space>Process<space>Features<apos><lf><space><space><space><space>transformer<space>parameter.<lf><space><space><space><space><quote><quote><quote><lf><lf><space><space><space><space>def<space>__init__<openparen>self<closeparen>:<lf><space><space><space><space><space><space><space><space><quote><quote><quote>Base<space>constructor<space>for<space>class<space>members.<quote><quote><quote><lf><space><space><space><space><space><space><space><space>pass<lf><space><space><space><space><space><space><space><space><lf><space><space><space><space>def<space>has_support_for<openparen>self<comma><space>support_type:<space>int<closeparen>:<lf><space><space><space><space><space><space><space><space><quote><quote><quote>This<space>method<space>is<space>called<space>by<space>FME<space>to<space>determine<space>if<space>the<space>PythonCaller<space>supports<space>Bulk<space>mode<comma><lf><space><space><space><space><space><space><space><space>which<space>allows<space>for<space>significant<space>performance<space>gains<space>when<space>processing<space>large<space>numbers<space>of<space>features.<lf><space><space><space><space><space><space><space><space>Bulk<space>mode<space>cannot<space>always<space>be<space>supported.<space><lf><space><space><space><space><space><space><space><space>More<space>information<space>available<space>in<space>transformer<space>help.<lf><space><space><space><space><space><space><space><space><quote><quote><quote><lf><space><space><space><space><space><space><space><space>return<space>support_type<space>==<space>fmeobjects.FME_SUPPORT_FEATURE_TABLE_SHIM<lf><space><space><lf><space><space><space><space>def<space>input_from<openparen>self<comma><space>feature:<space>fmeobjects.FMEFeature<comma><space>input_tag:<space>str<closeparen>:<lf><space><space><space><space><space><space><space><space><quote><quote><quote>This<space>method<space>is<space>called<space>for<space>each<space>feature<space>which<space>enters<space>the<space>PythonCaller.<space><lf><space><space><space><space><space><space><space><space>Processed<space>input<space>features<space>can<space>be<space>emitted<space>from<space>this<space>method<space>using<space>self.pyoutput<openparen><closeparen>.<lf><space><space><space><space><space><space><space><space>If<space>knowledge<space>of<space>all<space>input<space>features<space>is<space>required<space>for<space>processing<comma><space>then<space>input<space>features<space>should<space>be<space><lf><space><space><space><space><space><space><space><space>cached<space>to<space>a<space>list<space>instance<space>variable<space>and<space>processed<space>using<space>group<space>processing<space>or<space>in<space>the<space>close<openparen><closeparen><space>method.<lf><space><space><space><space><space><space><space><space><quote><quote><quote><lf><space><space><space><space><space><space><space><space>try:<lf><space><space><space><space><space><space><space><space><space><space><space><space>#<space>foo<space>=<space>1<solidus>0<lf><space><space><space><space><space><space><space><space><space><space><space><space>self.pyoutput<openparen>feature<comma><space><quote>Output<quote><closeparen><lf><space><space><space><space><space><space><space><space>except<space>Exception<space>as<space>exc:<lf><space><space><space><space><space><space><space><space><space><space><space><space>feature.setAttribute<openparen><quote>fme_rejection_code<quote><comma><space><quote>error<quote><closeparen><lf><space><space><space><space><space><space><space><space><space><space><space><space>feature.setAttribute<openparen><quote>fme_rejection_message<quote><comma><space>str<openparen>exc<closeparen><closeparen><lf><space><space><space><space><space><space><space><space><space><space><space><space>self.pyoutput<openparen>feature<comma><space><quote><lt>REJECTED<gt><quote><closeparen><lf><space><space><space><space><space><space><space><space><space><space><space><space><lf><lf><space><space><space><space>def<space>close<openparen>self<closeparen>:<lf><space><space><space><space><space><space><space><space><quote><quote><quote>This<space>method<space>is<space>called<space>once<space>all<space>the<space>FME<space>Features<space>have<space>been<space>processed<space>from<space>input<openparen><closeparen>.<quote><quote><quote><lf><space><space><space><space><space><space><space><space>pass<lf>",
                                "editor": "python",
                                "name": "PYTHONSOURCE",
                                "prompt": "Enter Text Data",
                                "required": true,
                                "showEditButton": false,
                                "showInlineEditor": true,
                                "showPrompt": false,
                                "supportedValueTypes": [
                                    "none"
                                ],
                                "type": "text",
                                "valueType": "stringEncoded"
                            }
                        ],
                        "prompt": "Parameters",
                        "required": false,
                        "type": "group",
                        "valueType": "string"
                    }
                ]
            },
            "inputPorts": [
                {
                    "name": "Input"
                }
            ],
            "outputPorts": [
                {
                    "name": "Output"
                },
                {
                    "attributesAdded": [
                        {
                            "name": "fme_rejection_code",
                            "type": "buffer"
                        },
                        {
                            "name": "fme_rejection_message",
                            "type": "buffer"
                        }
                    ],
                    "name": "<REJECTED>"
                }
            ],
            "upgradeWarning": "",
            "version": 1
        }
    ]
}

I made that last point into its own idea, since it would be an expansion on what has already been added to fix the current one.

• Add per-port close() and has_support_for() method to Python transformers Open
• 2 Votes

I’m pleased to let you know that we have added a rejected port to the PythonCaller via the reject_feature() method.

This is available in the upcoming release of 2024.2, you can try it out now in the beta: https://fme.safe.com/downloads/#beta