Using PythonCaller to Grant Privileges in Enterprise Geodatabase (SQL)

Hi Lindsay, you need to access the parameter value dictionary like here:

https://community.esri.com/community/open-platform-standards-and-interoperability/blog/2019/03/21/using-arcgis-pros-python-environment-in-etl-tools

Thanks so much for the responses!

I ended up going with David's approach, using the ListExploder.

The privileges are now being applied via the Python code, but it takes over 5 minutes to process each feature class.

The features are being written to an Amazon RDS from a desktop PC, so the whole process is inherently slower than with a local database, but 5 minutes to apply privileges to each feature class seems a little excessive...

I had read that the PythonCaller will process commands for every feature passed to it if you use a function and that you should use a Python class if processing per feature class.

Should I wrangle the above code into the class template?

My proof of concept workbench is set to only read 2 feature classes and, looking at the workbench, it certainly seems that only one record is being passed to the PythonCaller per feature class:

Alternatively, could I (should I) iterate through each feature class in a shutdown Python script?

Thanks again for all the help!

I'm suspecting that the bottleneck is with arcpy, not FME. In fact, the PythonCaller hardly does anything at all but to pass on parameters to arcpy. You can verify this by commenting out the two last lines of the PythonCaller.

Try disabling the PythonCaller and manually executing the arcpy.ChangePrivileges_management calls from the ArcGIS Python console after FME has terminated and see how long it takes there.

Rewriting the PythonCaller to use a class definition rather than a function will have no impact on the performance in itself.

Hi David,

Thanks for the suggestion.

I ran a Python script to apply the privileges in isolation, and then ran the same process in ModelBuilder.

Interestingly (and disappointingly), the results are the opposite of what I had expected:

• ModelBuilder was fastest, applying privileges to both feature classes in 1 minute 12 seconds.
• The standalone Python script took 6 minutes 32 seconds.
• The FME workbench, running nearly identical code, took 9 minutes 28 seconds.

Given that ModelBuilder is essentially running the same Python code as the other two solutions, I can't figure out why there is such a large discrepancy.

I had hoped to use FME to automate a data upload process, currently being run through ModelBuilder, but given these results, I'm second guessing myself.

I think the overall process will still be faster in FME, but this is the first time I've ever found FME to be slower than an Esri product when performing a similar task, and it has left me scratching my head...

Logically, I find it very difficult to understand those findings (assuming that you're doing all the tests on the same geodatabase, with the same feature class containing the same data volume)., in particular the large discrepancy between the ModelBuilder and the standalone Python script. You may want to ask on the Esri forums about that, which could perhaps give some clues for why FME is even slower.

Hi David,

Yes, the data source and destination are the same in all three cases, so I can't understand my findings either.

This and some other behaviour makes me think there is something strange going on with our connection to the RDS, so I'm looking now at running the workbench on an AWS desktop, which should have much better throughput to the RDS.

Thanks once again for your assistance with the PythonCaller, I'll post an update here for completion once I find a solution to the performance issue.