Do jobs in self-hosted runners run in parallel? #26769

DaHuoKolmostar · 2020-03-06T06:07:42Z

DaHuoKolmostar
Mar 6, 2020

The document says "Jobs run in parallel by default. ". When I tried workflow in github-hosted runners, jobs do run in parallel. However, when I try to run the workflow in self-hosted runners, the jobs do not seem to run in parallel. I’ve tried in both macos and centos with the latest version runner v2.165.2.

Is it that jobs only run in parallel in github-hosted runners?

My work flow is as follows:

on: [push] jobs: Synthesize: runs-on: self-hosted steps: - uses: actions/checkout@v2 - run: | ./code_generation.sh - run: | ./basic_verification.sh - run: | ./synthesis.sh Estimation: runs-on: self-hosted needs: Synthesize steps: - name: check run: | pwd ls - name: start time run: | date - name: sleep run: | sleep 8 - name: end time run: | date Deeper_Verification: runs-on: self-hosted needs: Synthesize steps: - name: check run: | pwd ls - name: start time run: | date - name: sleep run: | sleep 8 - name: end time run: | date

And the runner output in CentOS is as follows:

2020-03-06 05:41:01Z: Listening for Jobs
2020-03-06 05:42:38Z: Running job: Synthesize
2020-03-06 05:42:46Z: Job Synthesize completed with result: Succeeded
2020-03-06 05:42:51Z: Running job: Deeper_Verification
2020-03-06 05:43:02Z: Job Deeper_Verification completed with result: Succeeded
2020-03-06 05:43:04Z: Running job: Estimation
2020-03-06 05:43:15Z: Job Estimation completed with result: Succeeded

Answered by weide-zhou

Mar 6, 2020

Hi @dahuokolmostar ,

Jobs should run in parallel for both hosted runner and self-hosted runner.

But for self-hosted runner, there are some comments:

One self-hosted runner can only run one job at a time, when no available runners are idle, the subsequent jobs will be in queueing until available runners are idle.
If have enough available self-hosted runners for a workflow run, each job will be assigned a self-hosted runner, and the jobs will run in parallel.
When multiple workflows are running, if the workflows are more than the available self-hosted runners, each workflow will be assigned only one self-hosted runner.

Thanks.

View full answer

weide-zhou · 2020-03-06T08:22:31Z

weide-zhou
Mar 6, 2020

Hi @dahuokolmostar ,

Jobs should run in parallel for both hosted runner and self-hosted runner.

But for self-hosted runner, there are some comments:

One self-hosted runner can only run one job at a time, when no available runners are idle, the subsequent jobs will be in queueing until available runners are idle.
If have enough available self-hosted runners for a workflow run, each job will be assigned a self-hosted runner, and the jobs will run in parallel.
When multiple workflows are running, if the workflows are more than the available self-hosted runners, each workflow will be assigned only one self-hosted runner.

Thanks.

2 replies

sebastiaanluca Feb 6, 2024

Well that's a bummer finding out after setting everything up 🥲

crohr Feb 16, 2024

If you are on AWS you might be interested in RunsOn, which handles the runner autoscaling for you.

allir · 2020-11-26T13:53:07Z

allir
Nov 26, 2020

Are there any plans to change this? That is so that a runner can run multiple jobs at the same time (concurrently).

It would be really nice, since a lot of the time the jobs are small and don’t require the full machine to run and could run multiple jobs at the same time. Plus most of the time the jobs are spun up in containers so just having fewer runner machines with multiple cpus to spin those up vs having multiple smaller machines or having to run multiple instances of the actions runner service per machine to achieve this…

For more cpu/resource intensive jobs you could select a runner via labels so that it would go onto a machine that is dedicated to that job?

0 replies

faddat · 2020-12-05T22:23:00Z

faddat
Dec 5, 2020

Fact is that currently, the GitLab runner is much better than the Github actions runnner.

Hopefully this will change.

1 reply

penso Feb 5, 2023

Yes, those are years apart, I was using gitlab and it's way better...

sohanyadav · 2021-08-03T14:01:25Z

sohanyadav
Aug 3, 2021

@allir i agree with you man!!!

0 replies

sohanyadav · 2021-08-03T14:02:58Z

sohanyadav
Aug 3, 2021

We can run Runner in docker and then we can run multiple runners on same server!!!

0 replies

ivan-klass · 2021-11-08T12:51:09Z

ivan-klass
Nov 8, 2021

Thanks @weide-zhou for the details. I think it makes a lot of sense to add these comments into the official documentation.
Before I’ve found this discussion I’ve realized that all our repo workflows share the same default workspace directory on self-hosted runners, and it really made my hair stands on end. I was searching the docs to find out whether it’s actually a problem and there was no answer.

0 replies

devbro1 · 2022-09-02T19:22:10Z

devbro1
Sep 2, 2022

if you want to have multiple runners on the same machine you can run them under different users:

useradd runner1
usermod -a -G docker runner1
usermod -a -G rvm runner1
usermod -a -G wheel runner1
sudo su runner1

cd ~

#from https://github.com/actions/runner/releases
mkdir actions-runner && cd actions-runner
curl -O -L https://github.com/actions/runner/releases/download/v2.296.1/actions-runner-linux-x64-2.296.1.tar.gz
tar xzf ./actions-runner-linux-x64-2.296.1.tar.gz

#from https://github.com/financeit/centah/settings/actions/runners/new?arch=x64&os=linux
./config.sh --url https://github.com/financeit/centah --token ??? --name runner1 --unattended 
sudo ./svc.sh install
sudo ./svc.sh start

0 replies

AllanOricil · 2023-01-17T12:19:32Z

AllanOricil
Jan 17, 2023

This doc should include this information.
"parallelization is only possible when multiple runners are available"

1 reply

srid Jan 28, 2024

The doc you linked to no longer contains this statement.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GitHub Community

Do jobs in self-hosted runners run in parallel? #26769

{{title}}

Replies: 8 comments 4 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Do jobs in self-hosted runners run in parallel? #26769

Replies: 8 comments · 4 replies

Replies: 8 comments 4 replies