Can someone please explain how I can capture the Errors in the log file created by FME Server using Python. From what I have read I am still not sure if the python has to go in the startup, shutdown or PythonCaller transformer to capture any errors while it is running. I am also not a python developer so would appreciate a full working script or an example Workspace if possible? I am using FME 2016.
Hi @johnm, this Shutdown Python Script collects all lines containing "|ERROR" from the log file and save them as a list called "errors".
# Shutdown Python Script Example
# Collect error lines from the log file.
import fme
errors = []
with open(fme.logFileName, 'r') as f:
errors = [line for line in f.readlines() if 0 <= line.find('|ERROR')]
Then, what do you want to do using the error lines? For example, if you want to overwrite the log file with only the error lines collected, add these lines after the script above.
# Overwrite the log file with only the error lines if there were one or more error lines.
if 0 < len(errors):
with open(fme.logFileName, 'w') as f:
f.writelines(errors)
takashi wrote:
Hi @johnm, this Shutdown Python Script collects all lines containing "|ERROR" from the log file and save them as a list called "errors".
# Shutdown Python Script Example
# Collect error lines from the log file.
import fme
errors = []
with open(fme.logFileName, 'r') as f:
errors = [line for line in f.readlines() if 0 <= line.find('|ERROR')]
Then, what do you want to do using the error lines? For example, if you want to overwrite the log file with only the error lines collected, add these lines after the script above.
# Overwrite the log file with only the error lines if there were one or more error lines.
if 0 < len(errors):
with open(fme.logFileName, 'w') as f:
f.writelines(errors)
# Startup Python Script
# Add a function that collects error messages to FME Log File module.
import fmeobjects
errors = []
def errorCollector(sevirity, message):
if sevirity == fmeobjects.FME_ERROR:
errors.append('%s\n' % message)
fmeobjects.FMELogFile().setCallBack(errorCollector)
# Shutdown Python Script
# Do something using the collected error messages.
# e.g. overwrite the log file with the error messages if there were one or more errors.
import fme
if 0 < len(errors):
with open(fme.logFileName, 'w') as f:
f.writelines(errors) takashi wrote:
Hi @johnm, this Shutdown Python Script collects all lines containing "|ERROR" from the log file and save them as a list called "errors".
# Shutdown Python Script Example
# Collect error lines from the log file.
import fme
errors = []
with open(fme.logFileName, 'r') as f:
errors = [line for line in f.readlines() if 0 <= line.find('|ERROR')]
Then, what do you want to do using the error lines? For example, if you want to overwrite the log file with only the error lines collected, add these lines after the script above.
# Overwrite the log file with only the error lines if there were one or more error lines.
if 0 < len(errors):
with open(fme.logFileName, 'w') as f:
f.writelines(errors)
Hi @takashi, I forgot to mention that I need to write the errors into an Oracle table but I am not sure how I could do this until I get some example output data. Also, as I had in another question, the LogMessageStreamer was what I was trying to use but found it difficult to get it to run at the end of the script to capture all the errors.
johnm wrote:
Hi @takashi, I forgot to mention that I need to write the errors into an Oracle table but I am not sure how I could do this until I get some example output data. Also, as I had in another question, the LogMessageStreamer was what I was trying to use but found it difficult to get it to run at the end of the script to capture all the errors.
johnm wrote:
Hi @takashi, I forgot to mention that I need to write the errors into an Oracle table but I am not sure how I could do this until I get some example output data. Also, as I had in another question, the LogMessageStreamer was what I was trying to use but found it difficult to get it to run at the end of the script to capture all the errors.
Vote up this idea :-) ErrorCatcher Transformer
Hi @takashi Thanks for the above, I will use the python method you suggested as the startup & shutdown scripts work fine. How do I write the error messages into an Oracle table?
johnm wrote:
Hi @takashi Thanks for the above, I will use the python method you suggested as the startup & shutdown scripts work fine. How do I write the error messages into an Oracle table?
I myself have never experienced that, so cannot help you unfortunately. See a relevant section in FME Server documentation, and ask help of other experts if necessary.
Although it should be theoretically possible to connect Oracle database using a Python script, it could be a little troublesome to configure the environment of FME Server. If I were you, I would try first the method that uses the FMEServerJobSubmitter and FMEServerLogFileRetriever, in order to avoid make FME Server maintenance more complex.
Reply
Rich Text Editor, editor1
Editor toolbars
Press ALT 0 for help
Helpful Members This Week
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.