Why a matrix step will be canceled if another one failed

Hey,

I have a small PHP Job as the following

phpunit:
    name: PHPUnit
    runs-on: ubuntu-latest
    services:
      mysql:
        image: mysql:5.7
        env:
          MYSQL_ALLOW_EMPTY_PASSWORD: false
          MYSQL_ROOT_PASSWORD: password
          MYSQL_DATABASE: laravel
        ports:
          - 3306
        options: --health-cmd="mysqladmin ping" --health-interval=10s --health-timeout=5s --health-retries=3
    strategy:
      matrix:
        php: [7.2, 7.3]
    steps:
      - uses: actions/checkout@v1
      - name: Install Composer dependencies
        run: composer install --no-progress --no-suggest --prefer-dist --optimize-autoloader
      - name: Prepare the application
        run: |
          cp .env.example .env
          php artisan key:generate
      - name: Run Migration
        run: php${{ matrix.php }} artisan migrate -v
        env:
          DB_PASSWORD: password

If the tests failed in PHP 7.2 the workflow will not process any tests in PHP 7.3

Screen Shot 2019-09-07 at 1.54.30 PM.png

This is an example of the issue https://github.com/linuxjuggler/laravel-mysql-test/commit/18c405d62df99a5573145b48ac1888bcd4b743f2/checks

3 Likes

This is a fail-fast behavior, if one job fails the workflow fails and all jobs are discarded. If you run tests and one fails you don’t need to run more tests, because you already know there is a problem.

This does not make any sense, especially that am testing the code with two different PHP version.

Everywhere, it’s common for your code to work on PHP 7.2 for example and fail on 7.3 or vice-versa.

The same when your code, for example, pass a test on Mac but fail on Linux, it means that your mac build has a problem not all your code.

So when my code fails in 7.2, it means I have a problem when using PHP 7.2 it does not mean by default that all my code will fail no matter which version of PHP am using.

Your code should be tested for all the matrixes, not just one of them!!!

2 Likes

You can turn off fail-fast behavior of your workflow see: https://help.github.com/en/articles/workflow-syntax-for-github-actions#jobsjob_idstrategyfail-fast

10 Likes

It would be great if we could specify matrix configurations that should be allowed to fail. This is useful for a few cases, such as integration testing against new versions of some base framework, WIP support for certain environments, etc. It’s possible to do this by manually controlling success()/fail() checks, but it’s not as nice as being able to define this at the matrix level. Travis’ documentation for their equivalent: https://docs.travis-ci.com/user/customizing-the-build/#rows-that-are-allowed-to-fail

6 Likes

I think (as I didn’t test it yet) you can use Contexts and expression syntax for GitHub Actions for that

I was able to achieve this with continue-on-error: https://help.github.com/en/github/automating-your-workflow-with-github-actions/workflow-syntax-for-github-actions#jobsjob_idstepscontinue-on-error

Add an expression to the field which reads from matrix variables to ignore failures, for example:

jobs:
build:
runs-on: ubuntu-18.04
strategy:
matrix:
ocaml: ["4.07", "4.08", "4.09"]
steps:
- name: Checkout
uses: actions/checkout@v1
- name: Build
run: make
env:
OCAML_VERSION: ${{ matrix.ocaml }}
continue-on-error: ${{matrix.ocaml == '4.09' }}
4 Likes