upload service with e-mail subscription
Hi! I'm working on an FME Server based Upload Service, that should give feedback to uploaders via e-mail. This works well if the underlying workspace runs successfully, however, if the workspace fails I'm having problems getting FME Server to communicate with the uploader via e-mail. I need to get FME Server to communicate details of the failure (read: log) etc in an e-mail, not to sysadm, but to the 'user' directly. Subscriptions like 'JobSubmitter_Email_JobFailure' only send e-mail to sysadm on failure. Anyone know how to do this?
Hi, its not really an answer but a thought. You could the users username parameter (FME_SECURITY_USER) and you would then need to match it to something that contained their email. You then use the FMEServerEmailGenerator to set the email. Pretty sure that would work.
Now having said that, if the user uploads a file and doesn't tick it, the workbench won't run at all and therefore won't send an email. There is request in about that one. A workaround, would be to have another wrapper workspace around that, and that would need to check that you are actually getting input before running. Its doesn't deal with your issue but it also something to watch out for as no email would be sent in those cases.
Thanks for thoughts tdavis - useful. Got this from Ken Bragg @ Safe:
Here is what I'm thinking.
- When you publish the workspace and register with whichever service runs it, you can click edit and select "Notify on Job Completion". This lets you pick a topic for success or failure. So you would pick a failure topic
- create another secondary workspace with a textline reader (reader parameter - Read Whole File at Once > yes). Publish this one and register with the Notification Service and have this one Subscribe to the failure topic.
So at this point if the first one fails it triggers the second one via the topic. In the second workspace the reader will give you information from the job that failed in json like this:
"id": "24",
"StatusNumber": "798006",
"timeFinished": "Thu-03-Jul-2014 03:12:59 PM",
"jobfailure_topic": "failure",
"LogFileName": "job_24.log",
"timeStarted": "Thu-03-Jul-2014 03:12:59 PM",
"requestKeyword": "JOB_SUBMITTER_SERVICE",
"StatusMessage": "Terminator: Aborting Translation as directed by mapping file. Message is 'TranslationTerminated'.",
"timeRequested": "Thu-03-Jul-2014 03:12:59 PM",
"logHome": "C:/apps/FMEServer//Logs"
So then you could use the FMEServerEmailGenerator to put together an email with the log etc.. But where do you get the email address? You could potentially log it out in the first workspace I guess we need to know when and how it is submitted.
Which prompted this response from myself:
Here's, in short, how the data upload portal works (all on FME Cloud):
Stage 1: User uploads a ZIP file + enters name, e-mail, org, telephone etc.
Workspace 1 (Data Streaming Service) logs the request to a table in PostGIS, table-key being its own JobID on FME Server. Then the workspace validates the contents of the ZIP-file, which contains (or should contain) an Excel-file, PDF-reports and SHAPE-files amongst others. If the validation is successful the workspace/FME Server posts some (geo)JSON back to the portal containing all the geometries submitted, and moves the ZIP-archive to FME_SHAREDRESOURCE_DATA. If the validation fails the JSON contains an error log ready for display.
Stage 2: If the ZIP was valid is stage 1, and the user accepted the geometries drawn out in the portal (from the JSON), then workspace 2 is called to load the data into a PostGIS schema. Workspace 2 is registered with the Job Submitter service and reads details of the "uploader"/user from the log-table in PostGIS (JobID from workspace 1 is passed on). The e-mail needed for feedback is in this log-table, and the table-key to get to the relevant row is the FME Server JobID from workspace1 in stage1. Workspace 2 contains an FMEServerEmailGenerator, and notifies a topic connected to an e-mail subscription protocol upon success and a default topic (JobSubmitter_Email_JobFailure) if it fails.
In either case - workspace 2 failing or not - I need to get FME Server to e-mail the "uploader"/user directly with details of the data-load, and with error message+log if workspace 2 fails.
I'm already passing on the JobID from workspace 1 via JSON/portal and into workspace 2. I'm also pulling the relevant e-mail address I need to use back out of the log-table in workspace 2, so I have that there. Will your solution work then do you think? Sounds like we're almost there but not quite.
Then Ken Bragg @ Safe nailed it with this addition:
In a nutshell what I suggest is this. When you publish Workspace 2 and register with the Job Submitter Service set the fail topic to something like "failuretopic". Then create a workspace for failure using that text line reader I described. When you publish the failure handling workspace register it with Notification Service to have it subscribe to the failure topic.
In the failure workspace you will get the job_id in a json block from the reader as I described yesterday, and you can use that to go to your table and get the email address. Then use the FMEServerEmailGenerator. (You will need a new an email subscriber setup)
Two concerns:
- sometimes a workspace can hang rather than fail and we can't really do anything to check for that, so testing is key
- how exactly is the second workspace called? You mentioned it's registered with the Job Submitter Service but for this "notify on failure" to work it has to be launched from the Job Submitter Service.
Reply
Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.