Skip to main content

Hi All, I want to extract detailed log messages from FME job running on FME Flow (Ver 2021.2.3). What options are available to extract this information along with basic job information like Workspace, Repository, user, source, start and end time. etc. 

Appreciate any feedback. 

Jay 

 

 

 

You can make use of the REST API in behind FME Flow - https://docs.safe.com/fme/html/FME_REST/apidoc/v3/#/

 

You specifically be looking under the transformations group. Likely a combination of these:
 

 


 

Thanks hkingsbury for your response. I made some good progress using transformation group, when I run this code, I get this 401 response, I am not sure what I am missing here. 

 

Response o401]>
{}

 

import requests

base_url = "https://fmeserver:8443/fmerest/v3"
token = "tert434645646efere3445435"

headers = {
"Authorization": f"Bearer {token}",
"Accept": "application/json"
}

try:
# Get all jobs
response = requests.get(f"{base_url}/transformations/jobs", headers=headers)
print(response)
jobs = response.json()
print(jobs)

# Get detailed information for each job
for job in jobs:
job_id = job 'id']
print(job_id)
job_details = requests.get(f"{base_url}/transformations/jobs/{job_id}", headers=headers)
print(job_details)
# Process or store the job details as needed
print(job_details.json())

except Exception as e:
# handle it
print (e)

 


The easiest way to do this would be to use the HTTPCaller in FME, you can then use a Web Connection and this will handle your authentication. The header should be:

headers = {
"Authorization": f"fmetoken token={token}",
"Accept": "application/json"
}

The two endpoints you have there don’t exist, they should be /transformations/jobs/completed and /transformations/jobs/id/{job_id}.

To get the log, you’d then need to call /transformations/jobs/id/{jobid}/log. This will give you the log for the job which you can then parse if needed. Bare in mind that the logs aren’t kept forever (job records are, but no logs). These are cleared based on you System Cleanup configuration


 Thanks hkingsbury for your pointers. I was able to come up with code that provides all the messages that I needed to generate. 

I really appreciate your help. 

import requests

base_url = "https://fmeserver:8443/fmerest/v3"
token = "6454574747ghfjhfurturturtu"

headers = {
"Authorization": f"fmetoken token={token}",
"Accept": "application/json"
}

try:
response = requests.get(f"{base_url}/transformations/jobs/completed", headers=headers)
data = response.json()

items = data 'items']
for item in items:
job_id = item 'id']
source_name = item 'sourceName']

job_details = requests.get(f"{base_url}/transformations/jobs/id/{job_id}/log", headers=headers)
data_job_details = job_details.json()

data_job_details_items = data_job_detailst'items']

for data_job_details_item in data_job_details_items:
status = data_job_details_item_'status']
if (status == 'WARN'):
message = data_job_details_item_'message']
print("%s - %s - %s" % (job_id, source_name, message))

except Exception as e:
print (e)

 

 

 

 


Reply