Extract Detailed Job messages from FME Flow Successful/Failed Jobs
Hi All, I want to extract detailed log messages from FME job running on FME Flow (Ver 2021.2.3). What options are available to extract this information along with basic job information like Workspace, Repository, user, source, start and end time. etc.
You specifically be looking under the transformations group. Likely a combination of these:
Thanks hkingsbury for your response. I made some good progress using transformation group, when I run this code, I get this 401 response, I am not sure what I am missing here.
try: # Get all jobs response = requests.get(f"{base_url}/transformations/jobs", headers=headers) print(response) jobs = response.json() print(jobs)
# Get detailed information for each job for job in jobs: job_id = job 'id'] print(job_id) job_details = requests.get(f"{base_url}/transformations/jobs/{job_id}", headers=headers) print(job_details) # Process or store the job details as needed print(job_details.json())
except Exception as e: # handle it print (e)
The easiest way to do this would be to use the HTTPCaller in FME, you can then use a Web Connection and this will handle your authentication. The header should be:
The two endpoints you have there don’t exist, they should be /transformations/jobs/completed and /transformations/jobs/id/{job_id}.
To get the log, you’d then need to call /transformations/jobs/id/{jobid}/log. This will give you the log for the job which you can then parse if needed. Bare in mind that the logs aren’t kept forever (job records are, but no logs). These are cleared based on you System Cleanup configuration
Thanks hkingsbury for your pointers. I was able to come up with code that provides all the messages that I needed to generate.