Question

Best Job Submitter service mode (sync | async) for submitting many jobs in a row?

  • 13 February 2024
  • 1 reply
  • 31 views
dms2
Rank
Badge +10

We have 2 job submitter services to validate the data they’re sent, write files to FME Server/Flow resource directory, write data to an Oracle database and send a response of the data validation and writing via an HTTPCaller.

As far as I know some users use some automated mechanism to submit a high number of jobs in a row to any of these services, which are called asynchronously.

At some point someone suggested to call these services synchronously instead, but we are unsure about what is the best service mode (sync vs async) for this scenario. Any idea? I’ve searched in FME Server/Flow documentation to no avail.

 

1 reply

hkingsbury
Rank
Userlevel 5
Badge +29

There’s not really a black and white answer, depends on a range of considerations. I’ve made some comments below, hopefully they help drive your decision!

  • If the caller is needing to do something once the FME job is complete, then it will need to be synchronous
  • If the client is waiting on a job to complete, but the server is busy and the job is queued, does that cause an issue?
  • If the caller doesn’t need to wait, then async is probably the better solution, bear in mind though jobs will then be waiting in the queue
    • Will this block other time critical jobs? Can be overcome with routing rules
  • How highly specced is your database and how big are the calls being made? If you’re going async the number of concurrent FME processes could be too many for the DB

 

Check out the Abley FME Firesides on Youtube: https://www.youtube.com/@ableyltd

Reply


Community Stats

28,731
Posts
109,535
Replies
36,568
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.