How to limit concurrent workflow runs

Hello,

I have a workflow that deploys my project on push to master. When I push several changes in rapid succession, this causes multiple simultaneous deployments to happen. This happens a lot when rapidly merging automated PRs, such as dependabot PRs – when I merge 6 dependabot PRs, I need to deploy only once; not 6 times. I need to restrict the number of concurrent workflow runs to 1 so that only a single deployment runs at a time, or a way to queue workflows to run after some “debounce” period. What options do I have?

Thanks!

Bryan

46 Likes

Hi,

Currently it’s NOT supported to limit the number of workflow runs if you merge multi PRs into same branch with pull request trigger.

5 Likes

Thanks for the reply. Is anything like this planned? This is wasting a lot of build minutes for me both in GH Actions and with my hosting provider, who charges me for build minutes used during deployments.

1 Like

I would like this feature too. There are already hard limits on concurrency so it’d be nice to see those limit controls made available to users so we can lower them even further.

https://help.github.com/en/actions/automating-your-workflow-with-github-actions/workflow-syntax-for-github-actions#usage-limits

There are workarounds for this, but ideally this is something the automation pipeline should handle without the risk of introducing potential race conditions and manual coordination.

2 Likes

I’m also curious about this feature. Configurating concurrency for each workflow programatically or could I defer to run jobs till previous workflow done? Or github have plan for this?

1 Like

Thanks all for your feedback!

I have submitted a query about limiting workflow runs number internally, will update if there’s a response.

4 Likes

Thanks all for your feedback!

Responsed from github:

This is something we’ll work on but we don’t have any timeline we can share yet.

4 Likes

any news on this? I too would like something like this

1 Like

Adding another use case: race conditions in deploy workflows

Assume I have a CD system that runs deploys on merges to master.

Two people merge quickly, the two actions will race and I won’t know which one will win.

The ability to limit concurrency (even if just to 1) would be great.

@jahed what workarounds did you have in mind?

7 Likes

Use a custom queue somewhere. Have workflows check the queue before executing. If there’s a running workflow, add to the queue and exit early. Workflows can either trigger the next run before exiting or something else can coordinate workflow runs.

Not saying it’s perfect but good enough until the feature is implemented.

1 Like

Adding one more vote for the addition of this feature - would really be helpful for the work we do!  

This is exactly as my use case. Would love that feature.

1 Like

While waiting for a first-party solution, you can actually achieve this using a shell script and Google Cloud Storage. By locking an object in a storage bucket, you can ensure that only one instance of a job runs at a time.

Here is a script that implements the underlying locking

https://github.com/marcacohen/gcslock#shell-script-usage

You’ll need to ensure that gsutil is available, and add lock and unlock steps to your workflow.

How do you unlock it if the GitHub Workflow fails?

2 Likes

I asked something similar not long ago https://github.community/t5/GitHub-Actions/Serializing-workflow-runs-in-the-context-of-continuous/m-p/45490

https://github.com/softprops/turnstyle is what I came up with

4 Likes

Any update on this? Is there a deadline, is it still planned, is it on hold?

2 Likes

We just use softprop’s turnstyle action. As the comment inside says - we wait for up to a minute for previous run to complete and abort the new run if previous one is not done by then.

jobs:

  # Wait for up to a minute for previous run to complete, abort if not done by then
  pre-ci:
    runs-on: ubuntu-latest
    timeout-minutes: 1
    steps:
      - name: 'Block Concurrent Executions'
        uses: softprops/turnstyle@v1
        with:
          poll-interval-seconds: 10
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

  ci:
    needs: pre-ci
    runs-on: ubuntu-latest
    steps:
      .....
2 Likes

This is a very common use case, and one that we’ve been trying to sort out as well. As others have stated, in a fast moving CD company, we often merge one PR after the next and the last thign we want is for that to result in n deployments.

I’m attempting to work around this by using semantic-release to create releases, then have a separate workflow that triggers on release and performs deploy. When multiple changes get merged into master at roughly the same time, only the latest one will successfully create a release since the others will fail due to there being new commits since the workflow run began.

It’s not perfect, but I anticipate it will work… If only I could get past this 500 error issue I’m getting on release workflows https://github.community/t5/GitHub-Actions/GitHub-Action-on-release-trigger-500-error-There-was-an-issue/m-p/52526#M8544

Good Day to all, any update about this? thanks

2 Likes