Question

How to call job_id log files from server to use summary info in message notifications?

  • 27 August 2018
  • 2 replies
  • 8 views

Badge

Hello everyone. I've been challenged with providing an FME Server workspace that parses all or some of the the Summary Stats info from the jobs log file in order to post it in a Slack and email notification.

My specific question is how can I ensure that the correct job_id log file is called once the workspace completes on the server?

These server workspaces are several that run independent of one another from the same repository to update a PostGIS table with updated records.

I currently have a workspace that successfully parses the Features Written and Number Records Added Summaries, but it's being ran against an older log file from the Desktop. I'd like to have Server recognize a new job id log file, call that file into this published workspace, parse the necessary info, write it to a MSWord template or templated email message, and send out as an update notification.

*I understand that Python is probably the best practice to perform this workflow, but outside of Python and using only my FME Pro Workbench, how can I achieve this using what's available in the Desktop and published to Server (FME Cloud) instance.

**My versions of all are 2018.1 Build 18520.


2 replies

Badge

Hi @tjpollard,

to run your workspace against all new log files you might want to schedule the job that is parsing the log files and keep track of which log files where processed during the last run. A simple way to do this would be to write out the job ids that have been processed to a SQLite DB with a FeatureWriter during your translation. Then you can check the SQLite at the beginning of the next translation to see which was the last job processed.

For the information, that you are getting from the log file, maybe also check out the FME Server REST API calls that allow you to check job status. The response also returns the number of output features besides the status (success/failure). Look for "Checking job status" on this page:

 

https://playground.fmeserver.com/using-the-rest-api/jobs/

Hope this helps!

Badge +1

I'm able to do this from within the workspace in workbench in a Shutdown Python Script. The workspace is published to FME Server hosted locally. Not FME Cloud. I'm trying to see if I can do the same from an FME server notification email.

 

import smtplib, fmeobjects, email.utils, re

from email.MIMEText import MIMEText

 

emailTo = ""

emailFrom = ""

 

smtpServer = "smtp.###.###"

 

FeaturesWritten = str(FME_FeaturesWritten)

 

featurelistformatted = ""

for item in FeaturesWritten.split(","):

featurelistformatted = featurelistformatted + item + "\\n"

 

FeaturesWritten = str(featurelistformatted)

FeaturesWritten = FeaturesWritten.replace("{", "")

FeaturesWritten = FeaturesWritten.replace("}", "")

FeaturesWritten = FeaturesWritten.replace(" GIS", "GIS")

FeaturesWritten = FeaturesWritten.replace("'", "")

FeaturesWritten = FeaturesWritten.replace(":", "\\t\\t")

FeaturesWritten = re.sub('L\\n', ' Features\\n', FeaturesWritten)

 

msg = MIMEText("Workspace Successful.\\n\\nThe following features were written:\\n" + "\\n" + FeaturesWritten + "\\n\\n")

 

msg["To"] = email.utils.formataddr(("Recipient", emailTo))

msg["From"] = email.utils.formataddr(("FME Server", emailFrom))

 

status = FME_Status

if status == 0:

msg["Subject"] = "Error - Load Failed"

else:

msg["Subject"] = "Loaded Successfully"

 

server = smtplib.SMTP(smtpServer)

 

try:

server.sendmail(emailFrom, emailTo.split(","), msg.as_string())

finally:

server.quit()

 

Reply