Question

Is there an easier way to send a summary report of successful workspace completion?

  • 29 October 2019
  • 3 replies
  • 20 views

Badge

I have a workspace that runs using FME Desktop and the windows task scheduler. I would like it to send out an email notification upon successful completion that includes a total record count, a list of ID of all the records, and a timestamp.

In my specific instances, I'm using a FeatureWriter to write lines of text to a number of text files that are named using fanout and a unique ID field. What I've come up with works but it feels clumsy. The screenshot below leaves out the actual FMW processing and starts with the Writer (bottom left) and another branch feeding in records.

Maybe you agree - there has got to be a better (more out of the box) way to do this, right? If so, I'm all ears.

Footnote: Most of my stuff runs in FME Server which utilizes notifications there (although not with the above type of summary), so I have never had to dig into bolting on things after your writer finishes up,

 


3 replies

Badge +5

I think what you're doing is basically correct but I would offer the following suggestions / comments:

- The FeatureWriter is guaranteed to not output any features on the Summary port until all features have been written, so your FeatureHolder+Sampler, following the FeatureWriter are not necessary

- From 2018 onwards, the FeatureWriter can be configured to allow all input features to also be output (i.e. enable the "One per Feature Type" Output Port option. If you do this, you shouldn't need a separate branch for your statistics calculation

- You can also get a record count for each written feature type from the FeatureWriter by making use of the "Feature Type List Attribute" Summary Attribute, so your StatisticsCalculator should not be necessary

If you use all those tips I think you should be able to reduce the complexity of your workspace considerably....

Badge +3

@agelfert

 

We stage the processes. After verification, move to next stage etc. (to prevent wrting incomplete succeses etc.)

Run the script and then parse the log file from the workbench.

I use regexp to extract relevant information. Then test it.

Here are some regexp of a old workspace I have:

([\\d-]+)\\s+([\\d\\:]+)([\\d\\|\\.\\s]+)(WARN)\\s+\\|((\\++)|(Attribute.*`(.*)'.*value\\s+`(.+)'))

which reads warnings from the log. (this is from a 2016 workspace)

You can use same technique to get all the stats.

 

Create report and send email/twitter or wahthavewegotmore.

-------------

You can also use bat files (though stringparsing is less powerfull), here is a sample part:

fme.exe "%Scripts%\\ARIS\\ARIS meldingen_v3 EnvPara.fmw"

FIND "Translation was SUCCESSFUL" "%Scripts%\\ARIS\\ARIS meldingen_v3 EnvPara.log" >nul

 

IF ERRORLEVEL 1 (

 

ECHO %TIME:~0,5% - ## FOUT: Aris MeldingDichtheidsmap update mislukt ## ^ >> "%Logfile%"

 

SET failed=yes

 

)

 

ELSE (

 

:: Log entry

 

ECHO !TIME:~0,5! - !Thema! update gereed... ^ >> "%Logfile%"

 

)
Badge

Great feedback. Thanks all! I have a whole bunch of workspaces to "fortify" with this kind of reporting. So I will try and incorporate your suggestions.

Reply