Solved

Python Shut down script- Building a URL including parameters

ciarab
Rank
Badge +9

Hi,

I was wondering could any of the python experts help me. I'm trying to write a python shut down script that calls FME Server and passes some parameter values into a workspace to run on Server.

I can successfully do the same on Desktop- run a workspace collect information in the shutdown script about featuresRead, featureswritten, timings and so on and kick off a new workspace using these values which records and reports on this.

So I was testing could I complete the same task on Server -- I am able to run the workspace on server as part of the shut down script but right now I am just populating all the parameter values with "Test".

#Building URL for FME Server# fmeURL='''http://SERVERFME03/fmejobsubmitter/Samples/samplepythonlogging.fmw?DestDataset_XLSXW=C%3A%5CFMESERVER_PRIME2%5Cshutdown%5Ctest_write.xlsx&Written;=test&Read;=test&FeaturesLogged;=TEST&Time;=TEST&maxmemory;=TEST&Status;=TEST&Error;=TEST&Location;=TEST&Workspace;=TEST&Build;=TEST&opt;_showresult=false&opt;_servicemode=sync '''

What is the correct syntax to replace "test" with the variables I have created regarding information about the translation-- eg fme.totalFeaturesRead etc. I have managed to do this through desktop but I just can't get the correct information to log when I test on Server, any advice would be appreciated as I am trying to learn python myself (slowly)

 

Thanks

icon

Best answer by ciarab 12 October 2017, 14:55

View original

10 replies

david_r
Rank
Userlevel 4

Why not simply use the FMEServerJobSubmitter transformer?

If you need it to run after all the writers have terminated, consider replacing the writers with FeatureWriters send all the summaries to an Aggregator (to reduce to one single feature), then the FMEServerJobSubmitter.

If you really do need to use Python, consider using the REST API rather than the jobsubmitter, it's easier (I think). I also recommend using the excellent requests module.

See here for more info: http://playground.fmeserver.com/using-the-rest-api/jobs/

jdh
Rank
Badge +22

Why not simply use the FMEServerJobSubmitter transformer?

If you need it to run after all the writers have terminated, consider replacing the writers with FeatureWriters send all the summaries to an Aggregator (to reduce to one single feature), then the FMEServerJobSubmitter.

If you really do need to use Python, consider using the REST API rather than the jobsubmitter, it's easier (I think). I also recommend using the excellent requests module.

See here for more info: http://playground.fmeserver.com/using-the-rest-api/jobs/

Probably because he needs the shut down global variables.

 

 

david_r
Rank
Userlevel 4
Probably because he needs the shut down global variables.

 

 

If it's the feature counts, you'll get them from the FeatureWriters.
ciarab
Rank
Badge +9
If it's the feature counts, you'll get them from the FeatureWriters.
Its just I'm trying to learn python and I have been tasked with doing this using python so I'm giving it a go despite my limited python skills..I'm learning anyway so I guess its not a bad task.

 

 

I just thought because I managed to do the same on desktop (fairly) easily I must be missing something simple on Server.

 

 

This was the info I was pulling out in Desktop

 

 

status = fme.status

 

errorMsg = fme.failureMessage

 

logFile = fme.logFileName

 

Read= fme.totalFeaturesRead

 

Written= fme.totalFeaturesWritten

 

licence= fme.licensing

 

Time=fme.cpuSysTime

 

memory=fme.memoryUsage

 

maxmemory=fme.peakMemoryUsage

 

FeaturesLogged=fme.numFeaturesLogged

 

Location=FME_MacroValues['FME_MF_DIR']

 

Workspace=FME_MacroValues['FME_MF_NAME']

 

Build=FME_MacroValues['FME_BUILD_NUM']

 

 

I'll take a look at your suggestion and see if I can get any further

 

 

david_r
Rank
Userlevel 4

Actually, I've been toying with an idea of making an open-source Python library for interacting with FME Server. The idea was to have an interface where you could write code like

myserver = FMEServer(host='http://myfmeserver')
myserver.get_token(username='admin', password='123', duration=1, scale='days')
my_params={'myparameter1': 'paramvalue1', 'myparameter2', 'param2'}
job_id = myserver.submit_job('my_repo', 'myworkspace.fmw', params=my_params)
status = myserver.get_job_status(job_id)

Is there any interest for something like this? 

What would you consider the most important functionality?

david_r
Rank
Userlevel 4
Its just I'm trying to learn python and I have been tasked with doing this using python so I'm giving it a go despite my limited python skills..I'm learning anyway so I guess its not a bad task.

 

 

I just thought because I managed to do the same on desktop (fairly) easily I must be missing something simple on Server.

 

 

This was the info I was pulling out in Desktop

 

 

status = fme.status

 

errorMsg = fme.failureMessage

 

logFile = fme.logFileName

 

Read= fme.totalFeaturesRead

 

Written= fme.totalFeaturesWritten

 

licence= fme.licensing

 

Time=fme.cpuSysTime

 

memory=fme.memoryUsage

 

maxmemory=fme.peakMemoryUsage

 

FeaturesLogged=fme.numFeaturesLogged

 

Location=FME_MacroValues['FME_MF_DIR']

 

Workspace=FME_MacroValues['FME_MF_NAME']

 

Build=FME_MacroValues['FME_BUILD_NUM']

 

 

I'll take a look at your suggestion and see if I can get any further

 

 

Just to turn the thing the thing upside-down: I'm guessing that you're calling a workspace to log the results from the first workspace?

 

If so, why not log it directly from the first workspace, would that be possible?
ciarab
Rank
Badge +9
Just to turn the thing the thing upside-down: I'm guessing that you're calling a workspace to log the results from the first workspace?

 

If so, why not log it directly from the first workspace, would that be possible?
Ha it would, but that would involve getting others involved as I have no admin rights I can't install ANYTHING without permission. So I don't have access to the correct packages to write to a database or excel or whatever. Might take a little while to get that sorted.

 

 

I thought it would be easier/quicker to just do it like this but doesnt look like the case

 

ciarab
Rank
Badge +9

Why not simply use the FMEServerJobSubmitter transformer?

If you need it to run after all the writers have terminated, consider replacing the writers with FeatureWriters send all the summaries to an Aggregator (to reduce to one single feature), then the FMEServerJobSubmitter.

If you really do need to use Python, consider using the REST API rather than the jobsubmitter, it's easier (I think). I also recommend using the excellent requests module.

See here for more info: http://playground.fmeserver.com/using-the-rest-api/jobs/

Think I figured it out anyway managed to get one value passed through I'll test the rest tomorrow.

 

 

 

 

david_r
Rank
Userlevel 4

Actually there's a decent example on how to submit a job on FME Server using Python here:

http://playground.fmeserver.com/python-request/

Notice that they're using Python 3.4 and not the default 2.7, however.

ciarab
Rank
Badge +9

Meant to post as this was the solution I ended up with- might not be the easiest way to go about logging shutdown information it but if anyone is ever looking to call FMEServer and pass in some parameters from python shutdown this is the code I used. It works and I learnt something along the way

#parameters#
status = fme.status
errorMsg = fme.failureMessage
logFile = fme.logFileName
Read= fme.totalFeaturesRead
Written= fme.totalFeaturesWritten
licence= fme.licensing
Time= fme.cpuSysTime
memory=fme.memoryUsage
maxmemory=fme.peakMemoryUsage
FeaturesLogged=fme.numFeaturesLogged
Location=FME_MacroValues['FME_MF_DIR']
Workspace=FME_MacroValues['FME_MF_NAME']
Build=FME_MacroValues['FME_BUILD_NUM']


#Logging Call to the Logfile#


logger = open(FME_LogFileName,'a')
logger.write("http://OSIHFME03/fmejobsubmitter/Samples/pythonlogging.fmw?DestDataset_XLSXW=C%3A%5CFMESERVER_PRIME2%5Cshutdown%5Ctest_write.xlsx&"+"Written="+str(Written)+"&Read;="+str(Read)+"&Workspace;="+str(Workspace)+"&Status;="+str(status)+"&Build;="+str(Build)+"&Time;="+str(Time)+"&Error;="+str(errorMsg)+"&FeaturesLogged;="+str(FeaturesLogged)+"&Location;="+str(Location)+"&maxmemory;="+str(maxmemory)+"opt_showresult=false&opt;_servicemode=sync")
logger.close()


#Calling workspace on FME Server#
url= "http://MyFMEServer/fmejobsubmitter/Samples/pythonlogging.fmw?DestDataset_XLSXW=C%3A%5CFMESERVER_PRIME2%5Cshutdown%5Ctest_write.xlsx&"+"Written="+str(Written)+"&Read;="+str(Read)+"&Workspace;="+str(Workspace)+"&Status;="+str(status)+"&Build;="+str(Build)+"&Time;="+str(Time)+"&Error;="+str(errorMsg)+"&FeaturesLogged;="+str(FeaturesLogged)+"&Location;="+str(Location)+"&maxmemory;="+str(maxmemory)


webbrowser.open(url)

Reply


Community Stats

28,729
Posts
109,526
Replies
36,564
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.