Is it possible to require all GitHub Actions tasks to pass without enumerating them? #26733

jameslamb · 2020-06-14T00:45:18Z

jameslamb
Jun 14, 2020

The Question

Is it possible to configure required status checks in a repo to say “all GitHub Actions tasks must pass”, instead of needing to select individual GitHub Actions tasks by name?

This is how Travis, CircleCI, AppVeyor, and Azure DevOps integrations with GitHub all work…the default behavior is that all tasks must pass, and you can selectively mark tasks as optional.

I’d like to be able to have an administrator set this one time and not have to change it every time we change our GitHub Actions flow.

Background Context

Consider the screenshot below:

image924×284 40.3 KB

This is taken from a pull request in a project I’m working on. The master branch of that repository has a GitHub Actions workflow with 4 tasks. A repo administrator has manually marked each of them as required, following the steps in “About required status checks”. You can see, for example, that a task r-package (macOS-latest, clang) is required.

The pull request I’m proposing changes that GitHub Actions workflow, such that there is no longer a task called r-package (macOS-latest, clang). I am now in a situation where the pull request can’t be merged by maintainers because it looks like the required tasks never ran, and there a bunch of new non-required tasks that passed. See microsoft/LightGBM#3065 (comment) for more background.

Thanks for the time and consideration!

Answered by webknjaz

Dec 4, 2021

@graingert it was discovered that when dependencies don’t succeed, that check job is marked as skipped which auto-merge treats the same way as success. I solved this problem with https://github.com/re-actors/alls-green.

Here's a short example:

---

on:
  pull_request:
  push:

jobs:
  tests:
    ...
    strategy:
      matrix:
        python-version:
        # - ~3.12.0-0  # doesn't yet exist as of Oct 04, 2022
        - ~3.11.0-0
        - >-
          3.10
        - 3.9
    ...
    continue-on-error: >-  # this allows "unstable" versions to fail, except for 3.11, meaning 3.12 alpha when it's added
      ${{
        (
          startsWith(matrix.python-version, '~')
          && !startsW…

View full answer

msfjarvis · 2020-06-14T04:48:25Z

msfjarvis
Jun 14, 2020

Unfortunately there isn’t a way to do this. I’ve had this issue in the past as well where I had a path exclusion filter set up to not run build workflows on non-code changes and they similarly blocked merges. I had to end up doing this as a workaround.

1 reply

webknjaz Sep 24, 2022

@msfjarvis I've generalized this as https://github.com/marketplace/actions/alls-green#why a while back — maybe you could update your answer mentioning it?

Here's another answer from back then: https://github.com/orgs/community/discussions/26733#discussioncomment-3253155.

jameslamb · 2020-06-15T03:34:34Z

jameslamb
Jun 15, 2020
Author

oh sad. Yeah I guess it is just something we’ll have to deal with. Not a big deal! Overall I’m still really liking GitHub Actions.

Thanks for the help.

0 replies

graingert · 2020-09-03T23:00:37Z

graingert
Sep 3, 2020

@jameslamb I think another work around would be to create a job that “needs” the previous job:

jobs:
    test:
        matrix: ...
    allgood:
        needs: [test]

https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idneeds

but it costs an extra worker

0 replies

jameslamb · 2020-09-04T04:07:12Z

jameslamb
Sep 4, 2020
Author

oooo cool idea! That’s very clever, I like it. I think that would solve what I’m looking for, thank you.

0 replies

graingert · 2021-07-27T13:54:59Z

graingert
Jul 27, 2021

here’s the PR [ci] add an allgood job for easy [required] builds by graingert · Pull Request #3351 · microsoft/LightGBM · GitHub

0 replies

webknjaz · 2021-12-04T02:29:34Z

webknjaz
Dec 4, 2021

@graingert it was discovered that when dependencies don’t succeed, that check job is marked as skipped which auto-merge treats the same way as success. I solved this problem with https://github.com/re-actors/alls-green.

Here's a short example:

---

on:
  pull_request:
  push:

jobs:
  tests:
    ...
    strategy:
      matrix:
        python-version:
        # - ~3.12.0-0  # doesn't yet exist as of Oct 04, 2022
        - ~3.11.0-0
        - >-
          3.10
        - 3.9
    ...
    continue-on-error: >-  # this allows "unstable" versions to fail, except for 3.11, meaning 3.12 alpha when it's added
      ${{
        (
          startsWith(matrix.python-version, '~')
          && !startsWith(matrix.python-version, '~3.11')
        )
        && true
        || false
      }}
    ...
    steps:
    - ...

  check:  # This job does nothing and is only used for the branch protection
    if: always()

    needs:
    - tests

    runs-on: ubuntu-latest

    steps:
    - name: Decide whether the needed jobs succeeded or failed
      uses: re-actors/alls-green@release/v1
      with:
        jobs: ${{ toJSON(needs) }}

...

See https://github.com/marketplace/actions/alls-green#why.

10 replies

jameslamb Dec 23, 2022
Author

Ah! After adding the following to the workflow YAML:

if: always()

I saw the behavior you described...a failing required job caused the catch-all job to marked as failed, which blocked merging.

I know that if: always() was in the doc you linked, sorry for missing it.

I agree now, this action definitely solves the original issue I asked about more completely than what can be achieved with GitHub Actions' current behavior and the earlier suggestions in this discussion. I've modified this to mark your suggestion as the answer.

Thanks for your work on https://github.com/re-actors/alls-green and for the help!

webknjaz Dec 23, 2022

You're welcome :)

nikaro Jul 5, 2023

This is a very cool Action, thanks for that. But it works only for the jobs of one workflow. In the case where multiple workflows are triggered, and you want all of them to pass, currently there is no solution.

webknjaz Jul 5, 2023

@nikaro The solution is to use reusable workflows or have the action integrated into several workflows under different names and all added to the branch protection. I should probably document that.
I've also seen actions that allow polling to wait other checks so that's another way.
My preference is using the reusable workflows:

It allows to modularize different job definition, separating them logically even within the same repo
I came up with an approach of having such workflows prefixed with reusable- for visual distinction in the file list. They also only have workflow_call: trigger, no others.
I tend to call the main workflow definition ci-cd.yml and that's where I add alls-green. If it only consists of the other workflow calls, it's easier to reason about structuring the job dependencies on the high level. It also lets me avoid having a million lines of YAML in a single file.
Bonus: it's possible to make a separate workflow that runs on cron and include that ci-cd.yml (this is a good workaround for low-activity repos where the workflows with cron tend to become disabled periodically, so it wouldn't affect the PRs)

webknjaz Aug 8, 2023

@nikaro some of the above ideas are documented here: https://learn.scientific-python.org/development/guides/gha-basic/#reusable-workflows.

anmolchopra69 · 2023-04-10T19:36:52Z

anmolchopra69
Apr 10, 2023

How can we add/remove Details link ?
How can we re-direct to some url using Details link ?

1 reply

webknjaz Aug 8, 2023

That would require a full GitHub App implementation, that's beyond what GitHub Actions CI/CD can provide you with. Though, https://github.com/marketplace/actions/alls-green does print out some rich output via a Job Summary interface (it shows up on the workflow summary page, below the job relationships graph).

adamu · 2024-02-14T00:18:20Z

adamu
Feb 14, 2024

I run an app, Mergery, that does this.

5 replies

webknjaz Feb 15, 2024

Yes, some apps handle this properly. I think, Mergify does that too. However, this question seems to be more about GHA rather than external apps.

adamu Feb 15, 2024

Not at all. In fact if you use Mergery it requires no code to be committed to set up, and because it's an app it avoids the catch-22 of wanting all actions to pass, but doing that check itself in an action.

webknjaz Feb 16, 2024

Your statements seem to contradict each other, though.

adamu Feb 16, 2024

Good luck with your GH Action, man. I'm just sharing my solution here where people may find it, too. It's possible for multiple solutions to the same problem to exist in parallel, you do not need to criticize the ones that are different to your own. One day GitHub will probably make them all obsolete, anyway.

webknjaz Feb 16, 2024

I don't criticize, I pointed out above that such an app exists. And it's closed source too. Perhaps your wording confused me there.
It'd be nice to have this feature in GH, of course. Since it's not, my style is to make configs as code — straightforward and transparent. I know that apps can go beyond that since they can poll statuses from the old Statuses API as well as Checks API, which I didn't implement in the action, but maybe at some point I will.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GitHub Community

Is it possible to require all GitHub Actions tasks to pass without enumerating them? #26733

{{title}}

Replies: 8 comments 17 replies

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Is it possible to require all GitHub Actions tasks to pass without enumerating them? #26733

The Question

Background Context

Replies: 8 comments · 17 replies

jameslamb Jun 15, 2020 Author

jameslamb Sep 4, 2020 Author

jameslamb Dec 23, 2022 Author

Replies: 8 comments 17 replies

jameslamb
Jun 15, 2020
Author

jameslamb
Sep 4, 2020
Author

jameslamb Dec 23, 2022
Author