Question

Problem with python start up script (engine reboot) & problem with subprocess work around

11 years ago
June 6, 2013
5 replies
26 views

ebaiello
1 reply

Hello - I am having problems executing a python process as a startup script - the script accesses a series of ArcGIS modelbuilder models & iterates through many processes and consistently fails at around the 50th iteration (engine reboots). I researched and found the below link that suggested calling python with a subprocess as a work around for this behaviour. However, I can't get the script to run using the method in link below....Any ideas?

http://fmepedia.safe.com/articles/Error_Unexpected_Behavior/Shutdown-Python-Script-Causing-FME-Engine-Failure

2013-06-06 10:56:13| 0.2| 0.2|INFORM|Feature Manipulation Engine 2012 SP4 (20120926 - Build 12266 - WIN32)

2013-06-06 10:56:13| 0.2| 0.0|INFORM|FME_HOME is 'C:\\apps\\FMEServer\\Server\\fme\\'

2013-06-06 10:56:13| 0.2| 0.0|INFORM|FME Engine (floating)

2013-06-06 10:56:13| 0.2| 0.0|INFORM|Permanent License.

2013-06-06 10:56:13| 0.2| 0.0|INFORM|Machine host name is: sxmcifme001

2013-06-06 10:56:13| 0.2| 0.0|INFORM|START - ProcessID: 6404, peak process memory usage: 29372 kB, current process memory usage: 29272 kB

2013-06-06 10:56:13| 0.2| 0.0|INFORM|FME Configuration: Command line arguments are `fme' `NRM ETL Testing\\eba_17_7_TRAILS_STAGE_PROCESS\\eba_17_7_TRAILS_STAGE_PROCESS.fmw' `--FME_DATA_REPOSITORY' `//ds.fs.fed.us/efs/FS/Reference/GeoTool/agency/EDW/FME12Repository//Server/Upload/' `--SourceDataset_SDE30_3' `AllTrails' `--FME_SERVER_NAME' `Engine2' `-FME_ENGINE_MEMORY_REDLINE' `0.5' `-FME_SHARED_RESOURCE_DIR' `\\\\ds.fs.fed.us\\efs\\FS\\Reference\\GeoTool\\agency\\EDW\\FME12Repository\\Server\\repository\\NRM ETL Testing\\fmeresources' `-LOG_FILENAME' `//ds.fs.fed.us/efs/FS/Reference/GeoTool/agency/EDW/FME12Repository//Logs/jobsubmitterservice\\FME_312E2A06_1370537773439_6404.log' `--FME_SERVER_ROOT' `//ds.fs.fed.us/efs/FS/Reference/GeoTool/agency/EDW/FME12Repository/' `--DestDataset_GEODATABASE_SDE' `AllTrails_SDO' `--FME_SECURITY_ROLES' `fmeadmin fmeauthor fmeuser' `--FME_MF_NAME' `eba_17_7_TRAILS_STAGE_PROCESS.fmw' `--FME_SECURITY_USER' `ebaiello'

2013-06-06 10:56:13| 0.3| 0.1|INFORM|Python version 2.6 successfully loaded

2013-06-06 10:56:14| 0.5| 0.1|INFORM|FME_BEGIN_PYTHON: evaluating python script from string...

2013-06-06 10:56:14| 0.5| 0.0|WARN |Python Exception <SyntaxError>: invalid syntax (<string>, line 1)

2013-06-06 10:56:14| 0.5| 0.0|ERROR |Error executing string `Import subprocess

subprocess.call(['C:\\\\Python26\\\\ArcGIS10.0\\\\python.exe','C:\\\\EDW_Tools\\\\Roads_Trails\\\\Python\\\\TrailsSegmentation_12.py'])

2013-06-06 10:56:14| 0.5| 0.0|ERROR |FME_BEGIN_PYTHON failed to execute provided script

2013-06-06 10:56:14| 0.5| 0.0|INFORM|FME Session Duration: 0.6 seconds. (CPU: 0.3s user, 0.2s system)

2013-06-06 10:56:14| 0.5| 0.0|INFORM|END - ProcessID: 6404, peak process memory usage: 36824 kB, current process memory usage: 36668 kB

david_r
8321 replies
11 years ago
June 6, 2013

Hi,

could you please post your Python code here? It seems you have an invalid syntax somewhere, but it is difficult to tell when we can't see your code :-)

David

ebaiello
Author
1 reply
11 years ago
June 6, 2013

Hi David,

Here is the startup script calling sub process. And following that is the python script that does the work. Thanks for taking a look!

import subprocess

import pyfme

subprocess.call(['C:\\Python26\\ArcGIS10.0\\python.exe','//ds.fs.fed.us/EFS/FS/BusOps/CIO/Project/EDW/FME_Work/Roads_and_Trails/Processing_Tools/python/TrailsSegmentation_12.py'])

--------------------------------------------------------------------------------------

# Import arcpy module

import arcpy

import sys

import time

#Timer Object

from datetime import datetime

class TimerClass:

def __init__ (o):

o.sTime = datetime.now()

print "Timer Started at: " + str(o.sTime)

def pElapsed(o, text = ""):

dTime = datetime.now() - o.sTime

print text + str(dTime)

t = TimerClass()

dbConnections = "\\\\\\ds.fs.fed.us\\\\EFS\\\\FS\\\\BusOps\\\\CIO\\\\Project\\\\EDW\\\\FME_Work\\\\Roads_and_Trails\\\\Processing_Tools"

#dbConnections = "c:\\\\nrmpub\\\\additionalfiles\\\\roadsegmentation\\\\"

arcpy.env.workspace = dbConnections + "\\\\EDW_g_wo_nfs_eng_stage as fme.sde"

# Load required toolboxes

arcpy.ImportToolbox("\\\\\\ds.fs.fed.us\\\\EFS\\\\FS\\\\BusOps\\\\CIO\\\\Project\\\\EDW\\\\FME_Work\\\\Roads_and_Trails\\\\Processing_Tools\\\\EDW.tbx")

arcpy.gp.toolbox = "\\\\\\ds.fs.fed.us\\\\EFS\\\\FS\\\\BusOps\\\\CIO\\\\Project\\\\EDW\\\\FME_Work\\\\Roads_and_Trails\\\\Processing_Tools\\\\EDW.tbx";

#Empty AllTrails

arcpy.gp.EmptyAllTrails()

t.pElapsed("AllTrails features deleted : ")

tb = dbConnections + "\\\\EDW_g_wo_nfs_eng_stage as fme.sde\\\\EDW_SDE_SOURCE_CATALOG_V_temp"

rows = arcpy.SearchCursor(tb, "USE_IN_ETL = 'Y' AND SOURCE_FEATURE_CLASS = 'TRAIL'","","","schema")

lSchema = []

lForests = []

cnt = 0

for row in rows:

lSchema.append(row.schema)

lForests.append(row.forests)

cnt = cnt+1

del rows, row # Delete Cursor objects to remove locks

# Process Trails

for i in range(0,cnt):

print str(i) + " - " + lSchema[i] + " - " + lForests[i]

try:

if str(lForests[i]).find(",") == -1:

arcpy.gp.SegmentTrail(str(lForests[i]), str(lSchema[i]))

time.sleep (10)

t.pElapsed("processed "+ str(lSchema[i]) +" after : ")

#else:

fa = str(lForests[i]).split(',')

tq = ""

for j in range(0,len(fa)-1):

tq += "SECURITY_ID LIKE '" + fa[j] + "%' OR "

tq += "SECURITY_ID LIKE '" + fa[len(fa)-1] + "%' "

arcpy.gp.SegmentTrailCSA(tq, str(lSchema[i]))

print tq

t.pElapsed("processed CSA-"+ str(lSchema[i]) +" after : ")

except:

t.pElapsed(str(lSchema[i]) + " Failed after : ")

print "Unexpected error:", sys.exc_info()[1]

pass

t.pElapsed("Completed after : ")

david_r
8321 replies
11 years ago
June 6, 2013

Hi,

what happens if you execute the Python script from the command line, e.g.

C:\\Python26\\ArcGIS10.0\\python.exe .../TrailsSegmentation_12.py

At first glance it looks like the error reported by FME is caused by your external Python script, not FME itself.

David

david_r
8321 replies
11 years ago
June 6, 2013

Also try to replace all your backslashes "\\" with forward slashes "/" in the command path:

From

'C:\\Python26\\ArcGIS10.0\\python.exe'

'C:/Python26/ArcGIS10.0/python.exe'

Backslashes have a particular meaning in Python string literals, which can be confusing sometimes.

David

takashi
7597 replies
11 years ago
June 6, 2013

Hi, and also check indentations in the actual code. There are no indentations in the code you posted, structured indentations are very important in the Python code, invalid indentation causes a syntax error. Takashi

Reply

Rich Text Editor, editor1

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

Cookie settings

We use 3 different kinds of cookies. You can choose which cookies you want to accept. We need basic cookies to make this site work, therefore these are the minimum you can select. Learn more about our cookies.

Basic
Functional

Normal
Functional + analytics

Complete
Functional + analytics + social media + embedded videos + marketing

Problem with python start up script (engine reboot) & problem with subprocess work around