Skip to main content
  • EN

So I've had such a blessed, successful 24-hour run getting good advice on this forum, I'm going out on a limb and will keep it going...

 

As an FME Server shop, we leverage FME Server notifications quite a bit. Mostly the success/failure type with a custom template to inform the support team if things completed on schedule and whether everything is running normally. Show me when something is wrong and leave me alone the rest of the time. This is quite helpful.

 

In addition, I need more fine grained reporting for specific workbenches especially those powering integration with other teams. Oftentimes, that's basically an 'Emailer' transformer that sends out a custom message to custom audience. I posted about a related subject a while back. 

 

Now, I've recently put together a workspace that tries to gather various record counts across separate data streams. The easiest way I've found to do this is create any number of counters, starting at 0 and of type 'Local',

fme_counterthen have them feeding into a Sampler that grabs the last record it is fed,

 

fme_samplerand have all these streams feed into an aggregator that sums across each counter attribute.

 

fme_aggregatorThis works alright. But it really clutters up the workspace if you have several streams where you are counting ...

fme_counter_streamsI am always leery of workspaces with connections in spaghetti fashion. Makes it confusing and easy to mess up in drag ' drop fashion.

 

So long question short, is there a better way to gather this kind of information and feed an Emailer with it? Is there a way to get to my counter values without having them ride on a feature to the finish line? Could they be workspace parameter maybe? Any insights or experiences would be helpful. Thanks.

I don't know your exact data flow so this is just a suggestion. It may not apply to your needs.

Create an attribute to each stream that tells you what that stream is. For example: StreamType attribute that has the value 1 for stream 1 and 2 for stream 2.

Send all your streams to a StatisticsCalculator to get TotalCount with GroupBy set to that specific attribute (SteamType), instead of all the counters. Use the Summary port of the StatisticsCalculator so you get a single feature per Group (one for each stream if you configure it write), instead of aggregating (depending on what attributes you want in your data, using just the Summary port may not be viable).

For reporting purposes this type of data flow works great for most cases.

I don't know your exact data flow so this is just a suggestion. It may not apply to your needs.

Create an attribute to each stream that tells you what that stream is. For example: StreamType attribute that has the value 1 for stream 1 and 2 for stream 2.

Send all your streams to a StatisticsCalculator to get TotalCount with GroupBy set to that specific attribute (SteamType), instead of all the counters. Use the Summary port of the StatisticsCalculator so you get a single feature per Group (one for each stream if you configure it write), instead of aggregating (depending on what attributes you want in your data, using just the Summary port may not be viable).

For reporting purposes this type of data flow works great for most cases.

Hey great idea... hadn't thought of using the Statistics Calculator. I guess that still means routing a bunch of streams to a common reporting destination though. I'll play with it. Thanks for chiming in!!

Reply


Community Stats

30,179
Posts
114,557
Replies
37,844
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.