Can someone please explain how I can capture the Errors in the log file created by FME Server using Python. From what I have read I am still not sure if the python has to go in the startup, shutdown or PythonCaller transformer to capture any errors while it is running. I am also not a python developer so would appreciate a full working script or an example Workspace if possible? I am using FME 2016.
Question
Capturing Errors in FME Server Log files
This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
Hi @johnm, this Shutdown Python Script collects all lines containing "|ERROR" from the log file and save them as a list called "errors".
# Shutdown Python Script Example
# Collect error lines from the log file.
import fme
errors = []
with open(fme.logFileName, 'r') as f:
errors = [line for line in f.readlines() if 0 <= line.find('|ERROR')]
Then, what do you want to do using the error lines? For example, if you want to overwrite the log file with only the error lines collected, add these lines after the script above.
# Overwrite the log file with only the error lines if there were one or more error lines.
if 0 < len(errors):
with open(fme.logFileName, 'w') as f:
f.writelines(errors)
Hi @johnm, this Shutdown Python Script collects all lines containing "|ERROR" from the log file and save them as a list called "errors".
# Shutdown Python Script Example
# Collect error lines from the log file.
import fme
errors = []
with open(fme.logFileName, 'r') as f:
errors = [line for line in f.readlines() if 0 <= line.find('|ERROR')]
Then, what do you want to do using the error lines? For example, if you want to overwrite the log file with only the error lines collected, add these lines after the script above.
# Overwrite the log file with only the error lines if there were one or more error lines.
if 0 < len(errors):
with open(fme.logFileName, 'w') as f:
f.writelines(errors)
# Startup Python Script
# Add a function that collects error messages to FME Log File module.
import fmeobjects
errors = []
def errorCollector(sevirity, message):
if sevirity == fmeobjects.FME_ERROR:
errors.append('%s\n' % message)
fmeobjects.FMELogFile().setCallBack(errorCollector)
# Shutdown Python Script
# Do something using the collected error messages.
# e.g. overwrite the log file with the error messages if there were one or more errors.
import fme
if 0 < len(errors):
with open(fme.logFileName, 'w') as f:
f.writelines(errors) Hi @johnm, this Shutdown Python Script collects all lines containing "|ERROR" from the log file and save them as a list called "errors".
# Shutdown Python Script Example
# Collect error lines from the log file.
import fme
errors = []
with open(fme.logFileName, 'r') as f:
errors = [line for line in f.readlines() if 0 <= line.find('|ERROR')]
Then, what do you want to do using the error lines? For example, if you want to overwrite the log file with only the error lines collected, add these lines after the script above.
# Overwrite the log file with only the error lines if there were one or more error lines.
if 0 < len(errors):
with open(fme.logFileName, 'w') as f:
f.writelines(errors)
Hi @takashi, I forgot to mention that I need to write the errors into an Oracle table but I am not sure how I could do this until I get some example output data. Also, as I had in another question, the LogMessageStreamer was what I was trying to use but found it difficult to get it to run at the end of the script to capture all the errors.
Hi @takashi, I forgot to mention that I need to write the errors into an Oracle table but I am not sure how I could do this until I get some example output data. Also, as I had in another question, the LogMessageStreamer was what I was trying to use but found it difficult to get it to run at the end of the script to capture all the errors.
Hi @takashi, I forgot to mention that I need to write the errors into an Oracle table but I am not sure how I could do this until I get some example output data. Also, as I had in another question, the LogMessageStreamer was what I was trying to use but found it difficult to get it to run at the end of the script to capture all the errors.
Vote up this idea :-) ErrorCatcher Transformer
Hi @takashi Thanks for the above, I will use the python method you suggested as the startup & shutdown scripts work fine. How do I write the error messages into an Oracle table?
Hi @takashi Thanks for the above, I will use the python method you suggested as the startup & shutdown scripts work fine. How do I write the error messages into an Oracle table?
I myself have never experienced that, so cannot help you unfortunately. See a relevant section in FME Server documentation, and ask help of other experts if necessary.
Although it should be theoretically possible to connect Oracle database using a Python script, it could be a little troublesome to configure the environment of FME Server. If I were you, I would try first the method that uses the FMEServerJobSubmitter and FMEServerLogFileRetriever, in order to avoid make FME Server maintenance more complex.