Question

Subscribe workspace to multiple notifications 'AND'-style

  • 29 August 2016
  • 5 replies
  • 0 views
martinkoch
Rank
Badge +13

We are designing a conversion-job spanning multiple workspaces. We can run one after the other using notifications by subscribing the next workspace to the succes topic of the previous.

We could design this much more beautiful if we could e.g. start workspace 'E' and 'F' when 'B' and 'C' are ready. But when 'E' has a subscription to the succes-topics of 'B' and 'C' this is evaluated as an 'OR'. 'E' runs when 'B' or 'C' is ready.

It would be nice to evaluate the notifications a workspace is subscribed to with an 'AND' clause. You should do 'this', when you have 'this' and 'that'. Is there a possibility to do this using FME only? Ofcourse we could make our own 'notification service' using some database and python or C#, but as far as I can see, the only thing I am missing in FME-server is an 'AND' clause in the notification subscription.

With such possibility we can determine the prerequisites of every workspace and translate that into the success-topics to which the workspace should be subscribed.

We have multiple engines by the way, so it would speed up the total tranformationprocess if we could run independant workspaces simultaniously, and have workspaces depending on the results of multiple workspaces react on the finalisation of all previous.

Kind regards,

Martin

5 replies

patrick_koning
Rank
I would realy like to have an answer on this issue.

 

Could help in the current project.

 

 

itay
Rank
Badge +16

Hi @martinkoch,

The only way I can think of doing that is by using the workflow technique disscussed here.

Would love to know if there is another way to do it...

mark2atsafe
Rank
Userlevel 4
Badge +25

The other way that occurred to me is to use the Directory Watch protocol.

  • Workspace B and Workspace C should be set up to write a file to a directory (maybe use a shutdown script to write the file, so it won't occur until the workspace is done)
  • Create a Publication that watches for a "Modify" action on that file
  • That publication publishes to a topic that is watched by workspace E

If Workspace B finishes first, it will write the file, but as a "Create" action, so the publication won't fire. Then Workspace C finishes and writes the file as a "Modify" action. The publication fires and Workspace E starts.

Workspace E should probably delete that file so that it cleans up after itself and the process will be ready to go the next time.

I'm about 75% sure that the above method would work, and it doesn't seem too difficult to set up (just an extra layer of notifications). You could only AND 2 workspaces this way (ie you couldn't have B, C, and D write to the file) but you don't say whether that would be an issue or not.

I do really like the idea of an "AND" for notification subscribers, but I have no idea how difficult it would be and have a strong suspicion it would be quite complicated. Still, that's not our problem; it's for the developers to figure out!

I don't see an existing request for this, so I suggest you a) Log it with safe.com/support so an enhancement request is filed, and b) Add it as an idea on this site so that other users can vote for it if they feel it would be useful for them too

Hope this helps

Mark

itay
Rank
Badge +16

Hi, would be nice to have it integrated in server.

But until then you could use the workflow management system with notifications based on the level of processing.

So for example a chain of ws can be arranged to be triggered by notifications and dependancies between chains can be modeled via a controller ws calling on different chains.

martinkoch
Rank
Badge +13

The other way that occurred to me is to use the Directory Watch protocol.

  • Workspace B and Workspace C should be set up to write a file to a directory (maybe use a shutdown script to write the file, so it won't occur until the workspace is done)
  • Create a Publication that watches for a "Modify" action on that file
  • That publication publishes to a topic that is watched by workspace E

If Workspace B finishes first, it will write the file, but as a "Create" action, so the publication won't fire. Then Workspace C finishes and writes the file as a "Modify" action. The publication fires and Workspace E starts.

Workspace E should probably delete that file so that it cleans up after itself and the process will be ready to go the next time.

I'm about 75% sure that the above method would work, and it doesn't seem too difficult to set up (just an extra layer of notifications). You could only AND 2 workspaces this way (ie you couldn't have B, C, and D write to the file) but you don't say whether that would be an issue or not.

I do really like the idea of an "AND" for notification subscribers, but I have no idea how difficult it would be and have a strong suspicion it would be quite complicated. Still, that's not our problem; it's for the developers to figure out!

I don't see an existing request for this, so I suggest you a) Log it with safe.com/support so an enhancement request is filed, and b) Add it as an idea on this site so that other users can vote for it if they feel it would be useful for them too

Hope this helps

Mark

Thanks Mark2AtSafe. Using the filesystem as the 'memory' needed for an 'AND' put us on the right creative track.

 

 

I don't know if B or C will finish first, and I have workspaces depending on three others, but we'll try something with having the workspace 'E' check for files created at the end of B and by C. If they both exist, both workspaces have finished and E should run. This way I can subscribe E to both notifications of B and C. Only on the last notification E will find both 'markerfiles' and continue operation.

 

 

I will put up a suggestion for 'AND' and file this to support.

 

 

Kind regards,

 

 

Martin

Reply


Community Stats

28,688
Posts
109,430
Replies
36,542
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.