Skip to main content
Solved

Python Shutdown Script works in Desktop but not on Server


mmccart
Contributor
Forum|alt.badge.img+16

I have a python shutdown script that works fine in FME Desktop but when I try to run it in FME Server it fails. The logs aren't showing me anything useful.

I think the error may be due to my file path that I am trying to access. If I am trying to access a file via Python on the server, shouldn't I be using: r"//servername/directory/filename.cfg"?

Or should I try something else?

Thanks,

 

Mark

Best answer by mmccart

Resolved! Thanks to @SteveAtSafe's patience today, we figured out what the issue was.

First off, at some point during my testing, I had copied a version of the .py file to my FME Desktop installation folder (<FME Desktop>\\python\\python27\\). I also was using an old file name for the .py file as well in the directory. Because of this, any changes I was making to my .py file on my C:\\TEMP\\ directory was being ignored.

We fully tested loading the .py and .cfg file up FME Server to RESOURCES>ENGINE>PLUGINS>PYTHON via the Web UI and it ran successfully. We did have to change the path for the .cfg file in the Shutdown Python Script of the workflow to this: C:\\ProgramData\\Safe Software\\FME Server\\resources\\engine\\Plugins\\python\\ for it to work on the server.

Thank you again for everyone's help ( @larry, @david_r and @SteveAtSafe)

-Mark

View original
Did this help you find an answer to your question?

18 replies

mmccart
Contributor
Forum|alt.badge.img+16
  • Author
  • Contributor
  • August 10, 2016

 

Note: I put the necessary files in a directory on the FME server.

Forum|alt.badge.img
  • August 10, 2016

If you use a UNC path, make sure that the account running the FME Server Engines service has required permissions to access this folder. If it is the Local System account, try with a local path e.g. C:/.../...


david_r
Evangelist
  • August 10, 2016

@larry is right, if you need to access network resources (typically using UNC paths), you will need to run the engines as a domain user, as documented here: https://docs.safe.com/fme/html/FME_Server_Documentation/Content/AdminGuide/Running_FME_Server_Engines_Under_Different_Account.htm

If you refer to local files using a drive letter, then this isn't necessary, e.g. "C:/folder/file.ext" should work.


mmccart
Contributor
Forum|alt.badge.img+16
  • Author
  • Contributor
  • August 10, 2016
larry wrote:

If you use a UNC path, make sure that the account running the FME Server Engines service has required permissions to access this folder. If it is the Local System account, try with a local path e.g. C:/.../...

I have confirmed that the account running the FME Server Engine is a local admin on the machine and has full access to the folder in question. I tried putting the files in C:\\Temp\\ on the server and change the necessary code in the shutdown script to reflect that change. I then republished back to FME Server and it still failed.

 

 


mmccart
Contributor
Forum|alt.badge.img+16
  • Author
  • Contributor
  • August 10, 2016
david_r wrote:

@larry is right, if you need to access network resources (typically using UNC paths), you will need to run the engines as a domain user, as documented here: https://docs.safe.com/fme/html/FME_Server_Documentation/Content/AdminGuide/Running_FME_Server_Engines_Under_Different_Account.htm

If you refer to local files using a drive letter, then this isn't necessary, e.g. "C:/folder/file.ext" should work.

I have confirmed that the account running the FME Server Engine is a local admin on the machine and has full access to the folder in question. I tried putting the files in C:\\Temp\\ on the server and change the necessary code in the shutdown script to reflect that change. I then republished back to FME Server and it still failed.

 


Forum|alt.badge.img
  • August 10, 2016
mmccart wrote:
I have confirmed that the account running the FME Server Engine is a local admin on the machine and has full access to the folder in question. I tried putting the files in C:\\Temp\\ on the server and change the necessary code in the shutdown script to reflect that change. I then republished back to FME Server and it still failed.

 

 

Can you attach the log file please?

 

 


mmccart
Contributor
Forum|alt.badge.img+16
  • Author
  • Contributor
  • August 10, 2016
larry wrote:
Can you attach the log file please?

 

 

Here it is, though it doesn't provide much insight.

 

 

job-1814.txt

 

 


Forum|alt.badge.img
  • August 10, 2016
mmccart wrote:
Here it is, though it doesn't provide much insight.

 

 

job-1814.txt

 

 

In the FME Server web page, go in Manage->Resources and navigate to Logs>engine>current and open the XXX_fmepreocessmonitorengine.log.

 

 

Look at the time you ran the workspace and maybe you'll see some error about the END_PYTHON.

 

 

I forced a division by zero in the Shutwown script and here's what I see in the XXX_fmepreocessmonitorengine.log:

 

 

Wed-10-Aug-2016 03:38:16 PM   INFORM   Thread-14   KEPLER_Engine1   FME_END_PYTHON: evaluating python script from string...
Wed-10-Aug-2016 03:38:16 PM   INFORM   Thread-14   KEPLER_Engine1   Python Exception <ZeroDivisionError>: integer division or modulo by zero
Wed-10-Aug-2016 03:38:16 PM   INFORM   Thread-14   KEPLER_Engine1   Error executing string `a = 1
Wed-10-Aug-2016 03:38:16 PM   INFORM   Thread-14   KEPLER_Engine1   b = 0
Wed-10-Aug-2016 03:38:16 PM   INFORM   Thread-14   KEPLER_Engine1   c = a/b'
Wed-10-Aug-2016 03:38:16 PM   INFORM   Thread-14   KEPLER_Engine1   FME_END_PYTHON failed to execute provided script

 


mmccart
Contributor
Forum|alt.badge.img+16
  • Author
  • Contributor
  • August 10, 2016
larry wrote:

If you use a UNC path, make sure that the account running the FME Server Engines service has required permissions to access this folder. If it is the Local System account, try with a local path e.g. C:/.../...

@larry / @david_r, after looking at the log, I discovered this line:

 

Python Exception : No module named overwrite_hosted_features

 

 

This is the module I am trying to load. The file is located on the C drive of the FME server and I am using these lines of code to access it:

 

 

import sys

 

sys.path.append(r"C:\\TEMP\\OverwriteScript")from overwrite_hosted_features import run

 

 

config_file = r"C:\\TEMP\\OverwriteScript\\overwrite_AVL_Direct.cfg"

 

run(config_file)

 

Any suggestions?

david_r
Evangelist
  • August 11, 2016
mmccart wrote:
I have confirmed that the account running the FME Server Engine is a local admin on the machine and has full access to the folder in question. I tried putting the files in C:\\Temp\\ on the server and change the necessary code in the shutdown script to reflect that change. I then republished back to FME Server and it still failed.

 

Then it's probably something else. Have a look at the fmeprocessmonitorengine.log file, as described by larry, it should give you all the necessary details for debugging.

 

 


Forum|alt.badge.img
  • August 11, 2016
mmccart wrote:
@larry / @david_r, after looking at the log, I discovered this line:

 

Python Exception : No module named overwrite_hosted_features

 

 

This is the module I am trying to load. The file is located on the C drive of the FME server and I am using these lines of code to access it:

 

 

import sys

 

sys.path.append(r"C:\\TEMP\\OverwriteScript")from overwrite_hosted_features import run

 

 

config_file = r"C:\\TEMP\\OverwriteScript\\overwrite_AVL_Direct.cfg"

 

run(config_file)

 

Any suggestions?
If you have remote desktop access to your FME Server and are also an administrator on this server, connect to the server and do the following steps (taking care to use your accounts, paths and file names):

 

- Open a command prompt, go to the fmeengine.exe path and type the following commands (this will open a python environment in the fmeengine context)

 

 

 

-If you have an error, you have a permission issue.

 

 

-If not, run the same command with the account used to run the FME Engine Service

 

-First use the runas command, this will open an extra window to perform your tests

 

 

 

 

I have an import error because my fmeserverservice account does not have access to c:\\temp folder.

 

 

Let me know what you found after testing this.

 

 


mmccart
Contributor
Forum|alt.badge.img+16
  • Author
  • Contributor
  • August 11, 2016
mmccart wrote:
@larry / @david_r, after looking at the log, I discovered this line:

 

Python Exception : No module named overwrite_hosted_features

 

 

This is the module I am trying to load. The file is located on the C drive of the FME server and I am using these lines of code to access it:

 

 

import sys

 

sys.path.append(r"C:\\TEMP\\OverwriteScript")from overwrite_hosted_features import run

 

 

config_file = r"C:\\TEMP\\OverwriteScript\\overwrite_AVL_Direct.cfg"

 

run(config_file)

 

Any suggestions?
@larry I was able to load the module successfully with both my account the the domain admin account we're using to run FME Server. I am still getting the following error in the fmepreocessmonitorengine.log: Python Exception : No module named overwrite_hosted_features

 

 

I also confirmed that both accounts have full access to the C:\\TEMP\\directory in question.

 


steveatsafe
Safer
Forum|alt.badge.img+12
  • Safer
  • August 11, 2016

Are all environments are the same bit version? Looks like FME Server is 32bit.


mmccart
Contributor
Forum|alt.badge.img+16
  • Author
  • Contributor
  • August 11, 2016
steveatsafe wrote:

Are all environments are the same bit version? Looks like FME Server is 32bit.

@SteveAtSafe It's 64-bit: FME Server 2016.1 - Build 16492 - win64

 


Forum|alt.badge.img
  • August 11, 2016
mmccart wrote:
@larry I was able to load the module successfully with both my account the the domain admin account we're using to run FME Server. I am still getting the following error in the fmepreocessmonitorengine.log: Python Exception : No module named overwrite_hosted_features

 

 

I also confirmed that both accounts have full access to the C:\\TEMP\\directory in question.

 

So it seems that there is still an issue with the service account.

 

If you look at the services, can you confirm that the "FME Server Engine" service is running with the expected account and not only the "FME Server Core" service?

 

Note that mine is a local account and not a domain one.

 

 


mmccart
Contributor
Forum|alt.badge.img+16
  • Author
  • Contributor
  • August 11, 2016
larry wrote:
So it seems that there is still an issue with the service account.

 

If you look at the services, can you confirm that the "FME Server Engine" service is running with the expected account and not only the "FME Server Core" service?

 

Note that mine is a local account and not a domain one.

 

 

@larry Yes, the FME Server Engines is running as the domain account that I tested with. Not a local account. The domain account is in the Administrators group on the server. Also, I explicitly gave full control to the folder in question to that domain account. Does it matter that the username running job is different than this domain account? If I go to Manage>Jobs, the Username that runs it is the user that uploaded it to FME Server originally. Is this correct? I remember from previous versions that it only showed the FME Services account.


Forum|alt.badge.img
  • August 12, 2016
mmccart wrote:
@larry I was able to load the module successfully with both my account the the domain admin account we're using to run FME Server. I am still getting the following error in the fmepreocessmonitorengine.log: Python Exception : No module named overwrite_hosted_features

 

 

I also confirmed that both accounts have full access to the C:\\TEMP\\directory in question.

 

I don't think the username running the job matters.

 

Next step is to add an EnvironmentVariableFetcher transformer to your workspace and log the value of USERNAME and USERDOMAIN when your workspace is running on FME Server to make sure we are running with the expected account.

 

 

If nothing new is found, next step is to move your python code in the FME Server resources in Engine > Transformers folder.

 

If you have success loading the script from there, next step will be to find a place for your

 

config file (maybe also in the resources under Data in a subfolder created for your project).

 

To access this file from your python script, you can use the FME_SHARESRESOURCE_DATA macro.

 

 


mmccart
Contributor
Forum|alt.badge.img+16
  • Author
  • Contributor
  • Best Answer
  • August 16, 2016

Resolved! Thanks to @SteveAtSafe's patience today, we figured out what the issue was.

First off, at some point during my testing, I had copied a version of the .py file to my FME Desktop installation folder (<FME Desktop>\\python\\python27\\). I also was using an old file name for the .py file as well in the directory. Because of this, any changes I was making to my .py file on my C:\\TEMP\\ directory was being ignored.

We fully tested loading the .py and .cfg file up FME Server to RESOURCES>ENGINE>PLUGINS>PYTHON via the Web UI and it ran successfully. We did have to change the path for the .cfg file in the Shutdown Python Script of the workflow to this: C:\\ProgramData\\Safe Software\\FME Server\\resources\\engine\\Plugins\\python\\ for it to work on the server.

Thank you again for everyone's help ( @larry, @david_r and @SteveAtSafe)

-Mark


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings