Job queueing priority for self-hosted runners?

I am using multiple self-hosted runners. My workflow has 10 jobs with some of these jobs taking an hour to complete. If all of my self-hosted runners are busy, jobs queue up as expected. What is not expected is that jobs that were submitted as part of a later workflow run will sometimes get executed before jobs that are part of a different workflow submitted earlier. Is there any way to setup my workflow jobs to be processed in the order in which they were submitted? In other words, if there are currently queued jobs from an earlier workflow run I want all of those jobs to be processed prior to any jobs submitted as part of a later workflow run. Is there any mechanism to control this?

2 Likes

@bboles,

Currently, we seem have no easy ways to set the workflow runs start up follow the order of time that they were triggered. I also did not find any built-in settings option to do this.

As a workaround, you can try to setup a job that will always run before other jobs start up in the same workflow (see jobs.<job_id>.needs), and this job contains the following steps:

  1. Use the “Get a workflow run” endpoint to get the ‘workflow_id’ and the ‘run_number’ of current run.

  2. Use the “List workflow runs” endpoint to list all the ‘queued’ and ‘in_progress’ status runs of the same workflow.

  3. Set a loop, check the ‘run_number’ of the listed runs, if there are some run numbers are less than that of current run, these run are the previous runs of this run. Wait the previous runs to be completed.

  4. Stop the loop after all the previous runs completed. Current job completed, run other jobs in current run.

Thank you for the response @brightran. I am still evaluating how to handle this on our end and will keep you suggestion in mind. Cheers!