How to limit concurrent workflow runs

GitHub Actions
#6

Thanks all for your feedback!

I have submitted a query about limiting workflow runs number internally, will update if there’s a response.

4 Likes
#7

Thanks all for your feedback!

Responsed from github:

This is something we’ll work on but we don’t have any timeline we can share yet.

4 Likes
#8

any news on this? I too would like something like this

1 Like
#9

Adding another use case: race conditions in deploy workflows

Assume I have a CD system that runs deploys on merges to master.

Two people merge quickly, the two actions will race and I won’t know which one will win.

The ability to limit concurrency (even if just to 1) would be great.

@jahed what workarounds did you have in mind?

7 Likes
#10

Use a custom queue somewhere. Have workflows check the queue before executing. If there’s a running workflow, add to the queue and exit early. Workflows can either trigger the next run before exiting or something else can coordinate workflow runs.

Not saying it’s perfect but good enough until the feature is implemented.

1 Like
#11

Adding one more vote for the addition of this feature - would really be helpful for the work we do!  

#12

This is exactly as my use case. Would love that feature.

1 Like
#13

While waiting for a first-party solution, you can actually achieve this using a shell script and Google Cloud Storage. By locking an object in a storage bucket, you can ensure that only one instance of a job runs at a time.

Here is a script that implements the underlying locking

https://github.com/marcacohen/gcslock#shell-script-usage

You’ll need to ensure that gsutil is available, and add lock and unlock steps to your workflow.

#14

How do you unlock it if the GitHub Workflow fails?

2 Likes
#15

I asked something similar not long ago https://github.community/t5/GitHub-Actions/Serializing-workflow-runs-in-the-context-of-continuous/m-p/45490

https://github.com/softprops/turnstyle is what I came up with

4 Likes
Prevent parallel workflows?
#16

Any update on this? Is there a deadline, is it still planned, is it on hold?

2 Likes
#17

We just use softprop’s turnstyle action. As the comment inside says - we wait for up to a minute for previous run to complete and abort the new run if previous one is not done by then.

jobs:

  # Wait for up to a minute for previous run to complete, abort if not done by then
  pre-ci:
    runs-on: ubuntu-latest
    timeout-minutes: 1
    steps:
      - name: 'Block Concurrent Executions'
        uses: softprops/turnstyle@v1
        with:
          poll-interval-seconds: 10
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

  ci:
    needs: pre-ci
    runs-on: ubuntu-latest
    steps:
      .....
2 Likes
#18

This is a very common use case, and one that we’ve been trying to sort out as well. As others have stated, in a fast moving CD company, we often merge one PR after the next and the last thign we want is for that to result in n deployments.

#19

I’m attempting to work around this by using semantic-release to create releases, then have a separate workflow that triggers on release and performs deploy. When multiple changes get merged into master at roughly the same time, only the latest one will successfully create a release since the others will fail due to there being new commits since the workflow run began.

It’s not perfect, but I anticipate it will work… If only I could get past this 500 error issue I’m getting on release workflows https://github.community/t5/GitHub-Actions/GitHub-Action-on-release-trigger-500-error-There-was-an-issue/m-p/52526#M8544

#20

Good Day to all, any update about this? thanks

2 Likes
#21

See post referencing Turnstyle.

I added the action and it blocked until the already running action completed.

image
image1228×308 47.7 KB

Thanks @softprops . I’d consider this featured request solved.

Thanks Github for making Actions flexible enough to allow the community to help make Actions feature rich.

2 Likes
#22

If actions happen fast, there are a lot of room for two actions getting green light and starting at the same time.
Quite easy to reproduce with repository dispatch event and curl. Just do two at the same time, and you’ll get two running actions at the same time. Race condition, because there is no real lock acquired by the action, only checking if no actions run at current point in time.

1 Like
#23

Thank you @softprops for Turnstyle.

Unfortunately with one of my repositories I have many jobs (>40) and despite setting the poll-interval-seconds high I do hit GitHub API limits. I think that there is still value in a GitHub-official solution to this problem.

2 Likes
#24

edit: I misinterpreted the in progress view of my deployment actions. Turnstyle does in fact allow my staging and production deploys to run in parallel.

I think Turnstyle is a clever solution but only for certain deployment setups. It’s not general purpose. Our deployment workflow has parallel jobs for staging and production, but those are considered separate runs and Turnstyle serializes them.

We need either a workflow-level concurrency limit, a mutex primitive, or best of all a queue primitive (to avoid wasted minutes).

#25

Would also like this feature. With the Turnstyle solution, it can eat a lot of minutes waiting for a deployment which can take ~10mins to deploy especially if we have a backlog of >10 commits staged for deployment.