Status check for a matrix jobs

I build several subprojects using a job with the matrix strategy. The matrix is dynamic - depends on the branch used. All of those matrix jobs need to succeed for the code to be good to merge. But I fail to find a way how to check the composite status of the branches. Which direction would you recommend?

@edzis,

The matrix is dynamic - depends on the branch used.

Do you mean that you set different matrix configurations in the workflow on different branches?

But I fail to find a way how to check the composite status of the branches.

What do you mean about “the composite status of the branches”?
Do you mean that summarize the latest checks statuses of all the branches as a composite status?
If so, maybe you can try to use the repository_dispatch to trigger a workflow to summarize all the checks statuses.

If I missed something, feel free to tell me. And if possible, you also can share your repository with us, so that we can check more detailed configurations related to the workflow.

I’m using a job that needs matrix job to check all matrix jobs are successful. For example:

jobs:
  build:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        number: [0, 1, 2]

    steps:
    - run: echo ${{ matrix.number }}
  
  build_success:
    runs-on: ubuntu-latest
    needs: build
    steps:
    - run: echo Done!

In the example above, build_success job is only run after all build jobs are successful, so you can check on the result of build_success job.

Yes.

PR can be merged only if all of the jobs from the calculated matrix complete successfully. If a single job from the matrix fails, the PR merge should be blocked.

I don’t understand where and what would I dispatch and how would that help.

This gives false-positives - if one of matrix build jobs fail, the build_success is skipped, which is considered as a pass. See screenshot:

I might generate trivial artifacts from each running matrix job and have a job that checks the existence of those based on matrix, but that sounds over-engineered. Hoping to find something more simple.

@edzis ,

The checks of a PR generally include the jobs in the workflows related to the PR, and some checks generated from other GitHub Apps.
The related workflows can be:

  1. triggered by the events related to pull request, such as pull_request, pull_request_review, pull_request_review_comment and pull_request_target.

  2. triggered by push event when pushing new commits to the head branch (source branch) of the PR. The new commits push also triggers the workflows (if any) run on the activity type ‘synchronize’ of pull_request event if any.

Normally, the workflows run on other branches that not related to the PR will not affect to this PR.

If you want the related checks on the PR must be passed before the PR can be merged, you can set some branch protection rules, and set the related checks as ‘Required’. More details, see “Defining the mergeability of pull requests”.

@brightran I believe we are not on the same page. I believe it might help for me to share more insights about the setup. Here is a short description and the action setup is shared on https://gist.github.com/edzis/17602cb75eb6346b9096b7b53498a424

Conceptually we have a monorepo with multiple projects (customizers) and a shared codebase used by those projects.
We calculate a matrix based on branch name.
When we work on a specific project, we do the work in a branch named CLIENT-PROJECT/FEATURE. Only that project needs to be tested and deployed on dev - the matrix contains only that project and the dev environment.
When we work on generic features only the tests need to be run but there should be no deployment - the matrix job defines deploy = false and only the onlyTest job is run.
When a PR is merged into master we need to test and deploy all projects to dev - matrix includes all projects and the dev environment.
When a project needs to be deployed to staging or production, we push a staging/CLIENT-PROJECT or production/CLIENT-PROJECT branch - the matrix contains only that project and the specified environment.

Any ideas how I can check if all the jobs defined by the matrix have succeeded?

  • I can not specify the parent “Deploy” job as required because that is newer run from the perspective of merge guards.
  • I can not specify all the possible jobs resulting from the matrix as most of them are not even run in feature branches.
  • I can not specify another job that uses the build job as required because, if one of my matrix jobs fails, that job is a false positive - it is skipped which is considered as a pass by the merge guard.

Hi @edzis,
IIUIC you want to summarize a matrix job to a single check. This way, you can tick only one checkbox in the protected branch settings of the repository and later matrix entries will only change in the corresponding workflow file.

This is a workaround using a utility job (buildall) after your matrix job (build):

  buildall:
    if: ${{ always() }}
    runs-on: ubuntu-latest
    name: Build (matrix)
    needs: build
    steps:
      - name: Check build matrix status
        if: ${{ needs.build.result != 'success' }}
        run: exit 1

In addition, I think you should have continue-on-error: false on the build job (it’s the default).

Thanks @hodbn, that fixes my issue!

I was not aware of

if: ${{ always() }}

and

if: ${{ needs.build.result != 'success' }}

And I tried continue-on-error: false before with no success, but kept the current setup because when merging into master I don’t want to stop all projects when some fail.

1 Like