Skip to main content
  • EN

My workspace has a python startup script which calls a REST API and downloads and saves a zipped file geodatabase in zip file called out.zip. The zip file is then unzipped into a Downloads folder. However, because the zip file also contains a folder, and the folder name changes each time (out of my control), I don't know where the final file geodatabase will be located.

For example, when I run the code, the location will be something like this:

...\Downloads\20181018\QSC_Extracted_Data_20181018_103130077000-7024\data.gdb

The next time, the location will be:

..\Downloads\20181019\QSC_Extracted_Data_20181019_111958123000-8024\data.gdb

It's that folder right above the data.gdb which changes.

My workaround was to have a python scripted parameter which searches the date folder (20181019) for a file named data.gdb and return the path. However, it seems that the python scripted paramter is run at run-time prior to the download of the file geodatabase and so, when the translation begins, it can't find it.

Relevant code for the python scripted parameter:

import os
import fme
unzip_dir = os.path.join(fme.macroValues 'Download_Path'],fme.macroValuesh'Today'])
for dirpath, dirnames, filenames in os.walk(unzip_dir):
    if dirpath.endswith('data.gdb'):
        return dirpath 

The error I get is:

Unable to connect to the File Geodatabase at '.'. Make sure the correct filename was specified, and that the geodatabase wasn't saved with a newer version of ArcGIS than the one installed locally. The error number from ArcObjects is: '-2147024894'. The error message from ArcObjects is: {}

I believe the order is

  1. Scripted parameters (in the order in which they appear in the Navigator tree)
  2. Startup script
  3. Readers, Creators etc.

The user parameters, including scripted parameters are resolved before the startup scripts are run.

I believe the order is

  1. Scripted parameters (in the order in which they appear in the Navigator tree)
  2. Startup script
  3. Readers, Creators etc.

The user parameters, including scripted parameters are resolved before the startup scripts are run.

I should've tested before spreading misinformation!
I should've tested before spreading misinformation!
Oddly enough, that information is not covered in the otherwise excellent python and fme article. https://knowledge.safe.com/articles/706/python-and-fme-basics.html

 

 

@KenAtSafe perhaps the runtime order could be explicitly mentioned in part 1 of the tutorial? Especially since it covers startup scripts before scripted parameters.

 

 

Oddly enough, that information is not covered in the otherwise excellent python and fme article. https://knowledge.safe.com/articles/706/python-and-fme-basics.html

 

 

@KenAtSafe perhaps the runtime order could be explicitly mentioned in part 1 of the tutorial? Especially since it covers startup scripts before scripted parameters.

 

 

I agree, that would be a very worthwhile addition. @NatalieAtSafe

 

PS: Ken doesn't work for Safe anymore...
I should've tested before spreading misinformation!
The order of processes including not only user parameters configuration, startup process also log file opening/closing and shutdown process is very important. I think it should be documented somewhere in the FME official documentations, not only mentioned in an article related to Python scripting.

 

In my understanding, the order seems to be:
  1. Load workspace
  2. Configure user parameters (including scripted ones)
  3. Open log file
  4. Startup process (Which one does run first - Tcl or Python?)
  5. Perform translation
  6. Close log file
  7. Shutdown process (Which one does run first - Tcl or Python?)

 

The order of processes including not only user parameters configuration, startup process also log file opening/closing and shutdown process is very important. I think it should be documented somewhere in the FME official documentations, not only mentioned in an article related to Python scripting.

 

In my understanding, the order seems to be:
  1. Load workspace
  2. Configure user parameters (including scripted ones)
  3. Open log file
  4. Startup process (Which one does run first - Tcl or Python?)
  5. Perform translation
  6. Close log file
  7. Shutdown process (Which one does run first - Tcl or Python?)

 

There's also a slight discrepancy in order of processing of input/close methods with multiple PythonCreators.

 

 

If there is a reader present, the order is

 

 

PythonCreator_1 Input

 

PythonCreator_2 Input

 

Readers

 

PythonCreator_1 Close

 

PythonCreator_2 Close

 

 

which is as per the documentation.

 

 

If there is no Reader, then the order is

 

PythonCreator_1 Input

 

PythonCreator_1 Close

 

PythonCreator_2 Input

 

PythonCreator_2 Close

 

 

which if you were expecting the same order as the first case, could cause issues.

 

 

Not that I can think of a scenario where I would be mixing Input and Close methods without a reader present, but I am strongly in favour of documenting behaviour regardless.

 

I believe the order is

  1. Scripted parameters (in the order in which they appear in the Navigator tree)
  2. Startup script
  3. Readers, Creators etc.

The user parameters, including scripted parameters are resolved before the startup scripts are run.

Thanks @jdh. This is what I suspected. In the end, I just edited my startup script to move my geodatabase to the location that the parameter is looking.

 

There's also a slight discrepancy in order of processing of input/close methods with multiple PythonCreators.

 

 

If there is a reader present, the order is

 

 

PythonCreator_1 Input

 

PythonCreator_2 Input

 

Readers

 

PythonCreator_1 Close

 

PythonCreator_2 Close

 

 

which is as per the documentation.

 

 

If there is no Reader, then the order is

 

PythonCreator_1 Input

 

PythonCreator_1 Close

 

PythonCreator_2 Input

 

PythonCreator_2 Close

 

 

which if you were expecting the same order as the first case, could cause issues.

 

 

Not that I can think of a scenario where I would be mixing Input and Close methods without a reader present, but I am strongly in favour of documenting behaviour regardless.

 

I see and I believe. Good detective work.

 

 

I see and I believe. Good detective work.

 

 

I'm in the process of creating a training module for python scripting in FME. Writing the materials has caused me to challenge a bunch of my habits (ex FME macro values), go through the documentation materials with a magnifying glass and test a whole lot of what if scenarios.

 

 

Reply


Community Stats

30,159
Posts
114,480
Replies
37,807
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.