Hello,
let's say that I have 2 queues:
Default
Test
each of them has priority set to 5.
And I have 2 engines, each one has both queues assigned. Both engines are in use at the moment. In which order new jobs will be taken? I performed some tests and it wasn't taken based on jobid (this was my assumption).
Lets sayd that in queue there are jobs:
JobId
Queue
10
Default
11Test
12Default
13Default
14Test
15Default
Best answer by lauraatsafe
View original
Given your scenario, my assumption would be that the jobs are executed in the order they are submitted to the server. But I'd love to hear it if I'm wrong.
Given your scenario, my assumption would be that the jobs are executed in the order they are submitted to the server. But I'd love to hear it if I'm wrong.
@david_r well from what I can see it doesn't work this way but I can't find a pattern. It looks a bit random.
+26
This is a good question for @lauraatsafe or @jlutherthomas?
+6
Hi @witos,
Great question! In a scenario like yours where there is a backlog of queued jobs with equal priorities, they won't necessarily be run in order of job id. Having two queues there even with equal priorities will cause the order in which the jobs are processed to be a bit different.
Here’s a summary of how the logic works when FME Server Engines decide which jobs to run when working with two queues with equal priorities assuming two engines with both queues assigned to them (it is different when using queues with different priority values as highest priority always wins):
When an Engine starts up, it will check for jobs in the first queue in its list of assigned queues. The queue list will be in alphabetical order. So in your scenario, after a restart, an engine will always check to see if there is a job in the default queue and run that first.
After running a job from the Default queue, the engine will then check if there are any jobs in the Test queue and it will alternate between them as long as there are jobs in both queues.
So if an Engine starts up, runs a job on the Default queue, then finds there are no jobs in the Test queue, it will run another job from Default. At this point, the tally would be 2 jobs run from Default and 0 from Test. Now, if there is a backlog of jobs on the Server, the engine will favour taking jobs from the Test queue over the Default queue until the tally is balanced.
+6
This is a good question for @lauraatsafe or @jlutherthomas?
Thanks for the tag, @virtualcitymatt! Sorry I totally missed this until now.
This is a good question for @lauraatsafe or @jlutherthomas?
@lauraatsafe Thanks for that deep and detailed explanation.
I too am trying to understand the job order - but my situation is even more simple.
One engine, one queue. The order in which jobs are processed is not clear to me. It is not by submission order, or job number. I even set additional schedules within an automation to place certain jobs in the queue after others, but the later jobs can end up running before all the first set of workspaces have finished.
Moreover, automations scheduled to start an hour or two later, can even get some of their jobs completed before the first automation's already queued jobs, if the first series of jobs are still waiting in the queue.
I do not believe that I have done anything with priorities, at least I didn't do so intentionally.
Still looking for means to:
1 - submit jobs in a particular order
2 - find a listing that will sort or show jobs in the order in which they will be executed
+25
I too am trying to understand the job order - but my situation is even more simple.
One engine, one queue. The order in which jobs are processed is not clear to me. It is not by submission order, or job number. I even set additional schedules within an automation to place certain jobs in the queue after others, but the later jobs can end up running before all the first set of workspaces have finished.
Moreover, automations scheduled to start an hour or two later, can even get some of their jobs completed before the first automation's already queued jobs, if the first series of jobs are still waiting in the queue.
I do not believe that I have done anything with priorities, at least I didn't do so intentionally.
Still looking for means to:
1 - submit jobs in a particular order
2 - find a listing that will sort or show jobs in the order in which they will be executed
I think it's probably better to post this as a new question. You'll get far more responses that way. It's harder for everyone to find your comment in an existing thread like this one. But be sure to reference this question so we know that it's similar.
Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.
