Skip to main content
Solved

Does a python startup script execute before a python scripted parameter?


rchoucroun
Contributor
Forum|alt.badge.img+10

My workspace has a python startup script which calls a REST API and downloads and saves a zipped file geodatabase in zip file called out.zip. The zip file is then unzipped into a Downloads folder. However, because the zip file also contains a folder, and the folder name changes each time (out of my control), I don't know where the final file geodatabase will be located.

For example, when I run the code, the location will be something like this:

...\Downloads\20181018\QSC_Extracted_Data_20181018_103130077000-7024\data.gdb

The next time, the location will be:

..\Downloads\20181019\QSC_Extracted_Data_20181019_111958123000-8024\data.gdb

It's that folder right above the data.gdb which changes.

My workaround was to have a python scripted parameter which searches the date folder (20181019) for a file named data.gdb and return the path. However, it seems that the python scripted paramter is run at run-time prior to the download of the file geodatabase and so, when the translation begins, it can't find it.

Relevant code for the python scripted parameter:

import os
import fme
unzip_dir = os.path.join(fme.macroValues['Download_Path'],fme.macroValues['Today'])
for dirpath, dirnames, filenames in os.walk(unzip_dir):
    if dirpath.endswith('data.gdb'):
        return dirpath 

The error I get is:

Unable to connect to the File Geodatabase at '.'. Make sure the correct filename was specified, and that the geodatabase wasn't saved with a newer version of ArcGIS than the one installed locally. The error number from ArcObjects is: '-2147024894'. The error message from ArcObjects is: {}

Best answer by jdh

I believe the order is

  1. Scripted parameters (in the order in which they appear in the Navigator tree)
  2. Startup script
  3. Readers, Creators etc.

The user parameters, including scripted parameters are resolved before the startup scripts are run.

View original
Did this help you find an answer to your question?

9 replies

jdh
Contributor
Forum|alt.badge.img+28
  • Contributor
  • Best Answer
  • October 22, 2018

I believe the order is

  1. Scripted parameters (in the order in which they appear in the Navigator tree)
  2. Startup script
  3. Readers, Creators etc.

The user parameters, including scripted parameters are resolved before the startup scripts are run.


david_r
Celebrity
  • October 22, 2018
jdh wrote:

I believe the order is

  1. Scripted parameters (in the order in which they appear in the Navigator tree)
  2. Startup script
  3. Readers, Creators etc.

The user parameters, including scripted parameters are resolved before the startup scripts are run.

I should've tested before spreading misinformation!

jdh
Contributor
Forum|alt.badge.img+28
  • Contributor
  • October 22, 2018
david_r wrote:
I should've tested before spreading misinformation!
Oddly enough, that information is not covered in the otherwise excellent python and fme article. https://knowledge.safe.com/articles/706/python-and-fme-basics.html

 

 

@KenAtSafe perhaps the runtime order could be explicitly mentioned in part 1 of the tutorial? Especially since it covers startup scripts before scripted parameters.

 

 


david_r
Celebrity
  • October 22, 2018
jdh wrote:
Oddly enough, that information is not covered in the otherwise excellent python and fme article. https://knowledge.safe.com/articles/706/python-and-fme-basics.html

 

 

@KenAtSafe perhaps the runtime order could be explicitly mentioned in part 1 of the tutorial? Especially since it covers startup scripts before scripted parameters.

 

 

I agree, that would be a very worthwhile addition. @NatalieAtSafe

 

PS: Ken doesn't work for Safe anymore...

takashi
Influencer
  • October 22, 2018
david_r wrote:
I should've tested before spreading misinformation!
The order of processes including not only user parameters configuration, startup process also log file opening/closing and shutdown process is very important. I think it should be documented somewhere in the FME official documentations, not only mentioned in an article related to Python scripting.

 

In my understanding, the order seems to be:
  1. Load workspace
  2. Configure user parameters (including scripted ones)
  3. Open log file
  4. Startup process (Which one does run first - Tcl or Python?)
  5. Perform translation
  6. Close log file
  7. Shutdown process (Which one does run first - Tcl or Python?)

 


jdh
Contributor
Forum|alt.badge.img+28
  • Contributor
  • October 22, 2018
takashi wrote:
The order of processes including not only user parameters configuration, startup process also log file opening/closing and shutdown process is very important. I think it should be documented somewhere in the FME official documentations, not only mentioned in an article related to Python scripting.

 

In my understanding, the order seems to be:
  1. Load workspace
  2. Configure user parameters (including scripted ones)
  3. Open log file
  4. Startup process (Which one does run first - Tcl or Python?)
  5. Perform translation
  6. Close log file
  7. Shutdown process (Which one does run first - Tcl or Python?)

 

There's also a slight discrepancy in order of processing of input/close methods with multiple PythonCreators.

 

 

If there is a reader present, the order is

 

 

PythonCreator_1 Input

 

PythonCreator_2 Input

 

Readers

 

PythonCreator_1 Close

 

PythonCreator_2 Close

 

 

which is as per the documentation.

 

 

If there is no Reader, then the order is

 

PythonCreator_1 Input

 

PythonCreator_1 Close

 

PythonCreator_2 Input

 

PythonCreator_2 Close

 

 

which if you were expecting the same order as the first case, could cause issues.

 

 

Not that I can think of a scenario where I would be mixing Input and Close methods without a reader present, but I am strongly in favour of documenting behaviour regardless.

 


rchoucroun
Contributor
Forum|alt.badge.img+10
  • Author
  • Contributor
  • October 23, 2018
jdh wrote:

I believe the order is

  1. Scripted parameters (in the order in which they appear in the Navigator tree)
  2. Startup script
  3. Readers, Creators etc.

The user parameters, including scripted parameters are resolved before the startup scripts are run.

Thanks @jdh. This is what I suspected. In the end, I just edited my startup script to move my geodatabase to the location that the parameter is looking.

 


fmelizard
Safer
Forum|alt.badge.img+18
  • Safer
  • October 23, 2018
jdh wrote:
There's also a slight discrepancy in order of processing of input/close methods with multiple PythonCreators.

 

 

If there is a reader present, the order is

 

 

PythonCreator_1 Input

 

PythonCreator_2 Input

 

Readers

 

PythonCreator_1 Close

 

PythonCreator_2 Close

 

 

which is as per the documentation.

 

 

If there is no Reader, then the order is

 

PythonCreator_1 Input

 

PythonCreator_1 Close

 

PythonCreator_2 Input

 

PythonCreator_2 Close

 

 

which if you were expecting the same order as the first case, could cause issues.

 

 

Not that I can think of a scenario where I would be mixing Input and Close methods without a reader present, but I am strongly in favour of documenting behaviour regardless.

 

I see and I believe. Good detective work.

 

 


jdh
Contributor
Forum|alt.badge.img+28
  • Contributor
  • October 23, 2018
fmelizard wrote:
I see and I believe. Good detective work.

 

 

I'm in the process of creating a training module for python scripting in FME. Writing the materials has caused me to challenge a bunch of my habits (ex FME macro values), go through the documentation materials with a magnifying glass and test a whole lot of what if scenarios.

 

 


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings