Question

FME Cloud - send email when process starts. send email when process fails or completes with log file attached.


Badge +1

Hi FMEers, I'm new to Cloud and have been looking at the Automations section. I feel like what I want to do is really simple, but I'm not sure if it's something I set up in Desktop, automations, notifications (classic) or in the Job Submitter notification area?

 

I have my workspace in the Cloud. I've tested it, and it runs and outputs all my data (although it comes up with a failure message). This is why I want to have an email automatically sent to me when the process gets started via a server app by the end user and when it ends (or fails) along with the log file, so I can check if it has written out everything, but is outputting as a failure. I've attached the log file.

 

Thanks in advance for your help.


3 replies

Badge

Hi @epro_admin​, in terms of the job error, by any chance is this a Utility Network? Do you run into any issues with this same workspace from FME Workbench? When it comes to receiving a notification for a Server App failure with the job log, there are many ways of achieving it, but this is the method I would recommend that uses a variety of moving parts. Below is an explanation of how it works, and please find the attached FME Server Project which you can import into your own FME Server to test and modify:

 

When publishing the Server App workspace, on the last step where you register the services, you can click "Edit..." to set properties. Under Notify on Job Completion, you can select a Topic that will be notified if the workspace fails while running with that specific service. You'll want to create a topic specifically for this Server App workspace so you can select it in the service properties (this can be done in the FME Server web UI under Automations > Notifications (Classic) > Topics > New).

 

Once you have the topic set up and the workspace republished with the Notify on Failure configured, create a new Automation with an "FME Server Topic notified" trigger. Select the topic you just created. Now, whenever the workspace fails it will send JSON to that topic with information about the job such as the ID, a link to the job log, etc. Here's an example of the message that will be received at the trigger (when selecting the Event as JSON key): https://pastebin.com/ijVJ1tpG

 

In order to parse the JSON, a new workspace receives the Event as JSON key using a Text parameter, and flattens the JSON. The workspace ends with an Automations Writer which when used in an Automation passes attribute data into downstream actions as "keys". In the Automation, an Email Action takes those keys to fill this information into the email body before sending the notification.

 

I covered quite a few things here, but hopefully having the FME Server project to experiment with will make things clear!

Badge

Hi @epro_admin​, in terms of the job error, by any chance is this a Utility Network? Do you run into any issues with this same workspace from FME Workbench? When it comes to receiving a notification for a Server App failure with the job log, there are many ways of achieving it, but this is the method I would recommend that uses a variety of moving parts. Below is an explanation of how it works, and please find the attached FME Server Project which you can import into your own FME Server to test and modify:

 

When publishing the Server App workspace, on the last step where you register the services, you can click "Edit..." to set properties. Under Notify on Job Completion, you can select a Topic that will be notified if the workspace fails while running with that specific service. You'll want to create a topic specifically for this Server App workspace so you can select it in the service properties (this can be done in the FME Server web UI under Automations > Notifications (Classic) > Topics > New).

 

Once you have the topic set up and the workspace republished with the Notify on Failure configured, create a new Automation with an "FME Server Topic notified" trigger. Select the topic you just created. Now, whenever the workspace fails it will send JSON to that topic with information about the job such as the ID, a link to the job log, etc. Here's an example of the message that will be received at the trigger (when selecting the Event as JSON key): https://pastebin.com/ijVJ1tpG

 

In order to parse the JSON, a new workspace receives the Event as JSON key using a Text parameter, and flattens the JSON. The workspace ends with an Automations Writer which when used in an Automation passes attribute data into downstream actions as "keys". In the Automation, an Email Action takes those keys to fill this information into the email body before sending the notification.

 

I covered quite a few things here, but hopefully having the FME Server project to experiment with will make things clear!

Hi @Mark Warren​ , Thank you for your reply. I'm digesting it.

 

I'm working with gps tracklogs and waypoints. It's not a utility network.

 

In FME Workbench all is smooth. I tested it there before I published it into the Cloud.

 

Have to take quite a few steps back here and there's going to be a few more follow-up questions.. where/how do I get the server project you provided in Cloud or do I start with it in Desktop? (I wasn't able to open it in desktop, but I'm not sure where in the Cloud to put it so I can open it.)

 

When I try to create a new notification in Cloud, I've tried putting in my email address that is used for creating my FME account, but the system isn't liking it. I do have another account, but I'm logged out of it. I have tried using that email just in case. No go.

 

Is it ok to select notify for success and failure or is that going to cause a loop?

 

I'm still really new to this. Sorry for the basic questions!

Badge

Hi @Mark Warren​ , Thank you for your reply. I'm digesting it.

 

I'm working with gps tracklogs and waypoints. It's not a utility network.

 

In FME Workbench all is smooth. I tested it there before I published it into the Cloud.

 

Have to take quite a few steps back here and there's going to be a few more follow-up questions.. where/how do I get the server project you provided in Cloud or do I start with it in Desktop? (I wasn't able to open it in desktop, but I'm not sure where in the Cloud to put it so I can open it.)

 

When I try to create a new notification in Cloud, I've tried putting in my email address that is used for creating my FME account, but the system isn't liking it. I do have another account, but I'm logged out of it. I have tried using that email just in case. No go.

 

Is it ok to select notify for success and failure or is that going to cause a loop?

 

I'm still really new to this. Sorry for the basic questions!

Hi @elpinguino​, my apologies for the late response! The FME Server Project is meant to be imported into FME Cloud. The FME Server web interface will have a section named "Projects" where you can create/import/export project files. For steps on how to import the project file I shared, please follow our documentation here > To Import a Project.

 

For the notification, I assume you're referring to the Send Email (External Action) in the Automation? Could you please share the error messages you're getting when trying to validate the email account settings? And what type of email account is this (Gmail, Outlook, etc)?

 

It is possible to select both Notify For Success and Failure. This could only cause a loop if the notification triggers the workspace to rerun itself (i.e: with an Automation), but that wouldn't be involved in this workflow.

Reply