+18
I want to use the AzureServiceBusConnector to get the messages from the Queue.
But if there are no messages in the queue I want to end the FME Job after 1 minute. Is there a way to terminate the process only if the queue is still waiting.
If the AzureServiceBus connector reads the expected amount of messages I don't want to get an error, only if it's waiting for nothing.
Best answer by debbiatsafe
View original
+18
A colleague suggested starting a Job with a JobSubmitter and then with a Decelerator force ending the Job after 1 minute.
I want it to try and get a message from the queue and then end after X seconds with or without features coming out.
+18
Hello
In the AzureServiceBusConnector, it is possible to set the receive mode as batch or stream when receiving from a Queue or Subscription.
As for your original question, it may be possible to use a Decelerator, VariableSetter, and VariableRetriever to terminate a workspace after a minute if no messages are received.
The VariableSetter is attached to the Output port of the AzureServiceBusConnector. If a feature/message is retrieved, then the variable is set. After the time limit (controlled by the Decelerator from the Initiator port of the AzureServiceBusConnector transformer), the VariableRetriever retrieves the variable. If the variable attribute has no value, then no message was received and the feature is passed to Terminator.
Note that I do not have access to an Azure Service Bus connection to test, so this is all theoretical.
+18
Thank you Debbi,
It looked very promising but… the initiator will output first, then go to the decelerator, wait 10 seconds output to whatever comes next and then the AzureServiceBus output will start outputting.
Connector_3 adds 1 message to the cue and then Connector_2 tries to receive, but fails. Disabling the Decelerator gets the message out.
This was not what I expected at all from the Decelerator. Hoped it could also work as some kind of FeatureHolder. And would slow down many workbenches because the whole workbench is waiting for the Decelerator branch to finish first.
Everything I tried so far will first execute the branch it started and then start a next branch so the output branch will either be started and never output or will not be started at all.
And the initiator branch will be executed before the output branch will be started.
"In the AzureServiceBusConnector, it is possible to set the receive mode as batch or stream when receiving from a Queue or Subscription.”
Using batch mode my workbench will at least end when I predicted the amount of messages to be correct or lower. In stream mode i have to terminate my process always, and that will fill my FME Server log with a lot of failed messages on successful processes.
+18
Btw…
I really don't understand the use of streaming transformers when engine resources are scarce.
+33
How about submitting a scheduled job to stop this job in 1 minutes using another engine / queue before initiating the AzureServiceBusConnector? :D
Ah, I should read the entire topic before acting smart.
+18
Or just look at the server settings?
But it feels tricky to depend on this
My first layer of defense is counting on the age of the message:
As long as I did no miss to much triggers this would shorten the queue.
+18
Hello
I had considered the Running Job Expiry Time parameter initially.
A possible issue would be if your job was still in the middle of processing when the time limit is reached, as the job would be shut down immediately. However, this might not be an issue depending on your workflow.
+18
The risk of running to long is now minimized by the first “layer of defense”. But there is still a small change I get a trigger without having a message. So the second solution may be twice or three times the length of a normal run.
But without it there would be a risk the process will run for weeks unless it is noticed.
message → trigger → trigger
will result in a second process waiting on a message then:
message → trigger
will end that job but start the next one which will be again running up to the next message. So this needs to be terminated. But this should not be possible from the other side.
message → message → trigger
will result in a message waiting to be picked up
message → trigger
will result in finding the old message and picking that up as well.
This will for example be the case when FME Server is updating during the trigger event.
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.
