Skip to main content
Solved

Capture specific log messages to process in my workspace

salvaleonrp
Enthusiast
Forum|alt.badge.img+15

During runtime, I wanted to read the log file for messages that contain 'START - ProcessID, 'FME Session Duration', END - ProcessID' and a few other messages (in the red boxes) into my workspace so I can include each message as a feature in my translation.

0684Q00000ArEJTQA3.png

I can get most of this information from a shutdown script like below. However, I need the information back into my translation so I can write it to a database table that records translation metadata.

# Import modules
import fme, datetime
from os import path

# Get current date and time
dateToday = datetime.date.today().strftime('%Y/%m/%d')
timeNow = datetime.datetime.now().time().strftime('%H.%M.%S')

print(dateToday)
print(timeNow)
print(fme.processID)
print(fme.status)
print(fme.elapsedRunTime)
print(fme.totalFeaturesRead)
print(fme.totalFeaturesWritten)

 

Best answer by david_r

Here's a template that shows how to get the process id inside the translation workflow, it should be fairly easy to adapt for other information in the FME log file.

get_process_id.fmwt

Startup script:

import re
import fme

with open(fme.logFileName, 'r') as f:
   contents = f.read()
   global process_id
   match = re.search('ProcessID: (\d+)', contents)
   if match:
      process_id = match.groups()[0]
   else:
      process_id = 'UNKNOWN'

PythonCaller:

import fme
import fmeobjects

def get_process_id(feature):
    global process_id
    feature.setAttribute('process_id', process_id)

 

View original
Did this help you find an answer to your question?

4 replies

david_r
Evangelist
  • david_rEvangelist
  • Best Answer
  • January 6, 2020

Here's a template that shows how to get the process id inside the translation workflow, it should be fairly easy to adapt for other information in the FME log file.

get_process_id.fmwt

Startup script:

import re
import fme

with open(fme.logFileName, 'r') as f:
   contents = f.read()
   global process_id
   match = re.search('ProcessID: (\d+)', contents)
   if match:
      process_id = match.groups()[0]
   else:
      process_id = 'UNKNOWN'

PythonCaller:

import fme
import fmeobjects

def get_process_id(feature):
    global process_id
    feature.setAttribute('process_id', process_id)

 

salvaleonrp
Enthusiast
Forum|alt.badge.img+15
  • salvaleonrpEnthusiast
  • Author
  • Enthusiast
  • January 6, 2020
david_r wrote:

Here's a template that shows how to get the process id inside the translation workflow, it should be fairly easy to adapt for other information in the FME log file.

get_process_id.fmwt

Startup script:

import re
import fme

with open(fme.logFileName, 'r') as f:
   contents = f.read()
   global process_id
   match = re.search('ProcessID: (\d+)', contents)
   if match:
      process_id = match.groups()[0]
   else:
      process_id = 'UNKNOWN'

PythonCaller:

import fme
import fmeobjects

def get_process_id(feature):
    global process_id
    feature.setAttribute('process_id', process_id)

 

Thanks @david_r. However, this may not be able to read the parts of the log - e.g "FME Session Duration:" and "END: "after the PythonCaller transformer.  

david_r
Evangelist
  • david_rEvangelist
  • January 7, 2020
salvaleonrp wrote:

Thanks @david_r. However, this may not be able to read the parts of the log - e.g "FME Session Duration:" and "END: "after the PythonCaller transformer.  

The duration, memory use etc are already available in the fme module in the shutdown script, e.g.

print('Workspace duration was {} seconds'.format(fme.elapsedRunTime))

See: http://docs.safe.com/fme/2019.2/html/FME_Desktop_Documentation/FME_Workbench/Configuration/FME_END_PYTHON.htm

salvaleonrp
Enthusiast
Forum|alt.badge.img+15
  • salvaleonrpEnthusiast
  • Author
  • Enthusiast
  • January 7, 2020
david_r wrote:

The duration, memory use etc are already available in the fme module in the shutdown script, e.g.

print('Workspace duration was {} seconds'.format(fme.elapsedRunTime))

See: http://docs.safe.com/fme/2019.2/html/FME_Desktop_Documentation/FME_Workbench/Configuration/FME_END_PYTHON.htm

I captured that already in my shutdown scripts as shown on my question. I'll keep your solution as the best answer as I only need the ProcessID to bring the rest of the translation metadata from the log to my outputs. I wish there was a better way to do it as part of the translation or record the data in a database table from within the shutdown script. My python skills are not at that level. I can workaround this for now.

Thank you very much!

Reply


Helpful Members This Week

Community Stats

30,971
Posts
117,398
Replies
38,757
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.