***Note from Migration:***
Original Title was: Automations "Merge messages" action should synchronize/merge output from "Run a workspace" actions
Automations "Merge messages" action should synchronize/merge output from "Run a workspace" action jobs. For example, for cleanup operations or notifications following completion of multiple workspace jobs.
Hi @kweller, thank you so much for contributing to the FME community with your idea here!
To make sure that we understand correctly, what output are you interested in specifically? For example, Job ID?
And, carrying on with the example 'Job ID', how would you expect the message format? For example, CSV ("1,2,3") or some form of list (job.id{0}, job.id{1}, job.id{2})?
You might also be able to leverage User Keys when working with Automations, if, for example, it's the scenario of the 'Job ID' values that you need to preserve.
(See this image here: https://safe.my.salesforce.com/sfc/p/30000000ePES/a/4Q000000NNnO/cBKWPB.FxXMCkxBbPtzo3lFTU9555ULC81ymG7rWov8)
@ryanatsafe I'm aware of the User Keys. For my current use case, the output doesn't matter, only the synchronization to start a single job once all the other jobs started by the Automation on the merged actions are complete. However, a JSON message document aggregating (as a list) the Automations message outputs from all the merged/synced jobs) would make sense. One could extract anything one wanted from that.
My current workaround involves a separate Automation that runs periodically, polling for job counts on a specific queue, and does its cleanup processing only if that count is zero. It's kind of a ham-handed solution, but it works.
With the release of the Split Merge Block in Automations for FME Server 2021.1, I'll flip this idea to "Released". More information on Split Merge Block can be found in our documentation (https://localhost/fmeserver/help/WebUI/Automations-Actions/Merging-Messages.htm?Highlight=split%20merge) or in this knowledge article (https://community.safe.com/s/article/job-orchestration-with-automations). Let us know how you're using it and if you have any feedback!
@kweller Have you had a chance to try out the Split Merge Block, which is a more functional take on the Merge Action? This was released for FME Server 2021.1. I think the Split Merge Block really speaks to your initial idea of synchronizing output from multiple Workspace Actions. It would be great to hear your thoughts on the feature, particularly if you have a use case for aggregating all message outputs from merged jobs, as this is something we're aware of but haven't completely tackled.