Improve matrix exclusion feature

Hi,

Currently, we have a job matrix that looks like that:

matrix:
        db-image:
          - "mariadb_10.1"
          - "mariadb_10.2"
          - "mariadb_10.3"
          - "mariadb_10.4"
          - "mysql_5.6"
          - "mysql_5.7"
          - "mysql_8.0"
        php-version:
          - "5.6"
          - "7.0"
          - "7.1"
          - "7.2"
          - "7.3"
          - "7.4-rc"

What we want is to run our automated tests on all supported DB versions, and on all supported PHP versions, using following rules:

 - run tests on all DB versions with PHP 7.3,

 - run tests on all PHP versions using MariaDB 10.4.

We have to use following exclusion rules to do this.

exclude:
          # Exclude all DB images except "mariadb_10.4" for all PHP versions except "7.3".
          - {db-image: "mariadb_10.1", php-version: "5.6"}
          - {db-image: "mariadb_10.1", php-version: "7.0"}
          - {db-image: "mariadb_10.1", php-version: "7.1"}
          - {db-image: "mariadb_10.1", php-version: "7.2"}
          - {db-image: "mariadb_10.1", php-version: "7.4-rc"}
          - {db-image: "mariadb_10.2", php-version: "5.6"}
          - {db-image: "mariadb_10.2", php-version: "7.0"}
          - {db-image: "mariadb_10.2", php-version: "7.1"}
          - {db-image: "mariadb_10.2", php-version: "7.2"}
          - {db-image: "mariadb_10.2", php-version: "7.4-rc"}
          - {db-image: "mariadb_10.3", php-version: "5.6"}
          - {db-image: "mariadb_10.3", php-version: "7.1"}
          - {db-image: "mariadb_10.3", php-version: "7.0"}
          - {db-image: "mariadb_10.3", php-version: "7.2"}
          - {db-image: "mariadb_10.3", php-version: "7.4-rc"}
          - {db-image: "mysql_5.6", php-version: "5.6"}
          - {db-image: "mysql_5.6", php-version: "7.0"}
          - {db-image: "mysql_5.6", php-version: "7.1"}
          - {db-image: "mysql_5.6", php-version: "7.2"}
          - {db-image: "mysql_5.6", php-version: "7.4-rc"}
          - {db-image: "mysql_5.7", php-version: "5.6"}
          - {db-image: "mysql_5.7", php-version: "7.0"}
          - {db-image: "mysql_5.7", php-version: "7.1"}
          - {db-image: "mysql_5.7", php-version: "7.2"}
          - {db-image: "mysql_5.7", php-version: "7.4-rc"}
          - {db-image: "mysql_8.0", php-version: "5.6"}
          - {db-image: "mysql_8.0", php-version: "7.0"}
          - {db-image: "mysql_8.0", php-version: "7.1"}
          - {db-image: "mysql_8.0", php-version: "7.2"}
          - {db-image: "mysql_8.0", php-version: "7.4-rc"}

This is far too verbose and complex to read/maintain.

First, I tried to use nested objects inside matrix, but it does not exclude anything.

matrix:
        db:
          - {image-name: "mariadb_10.1", skippable: true}
          - {image-name: "mariadb_10.2", skippable: true}
          - {image-name: "mariadb_10.3", skippable: true}
          - {image-name: "mariadb_10.4"}
          - {image-name: "mysql_5.6", skippable: true}
          - {image-name: "mysql_5.7", skippable: true}
          - {image-name: "mysql_8.0", skippable: true}
        php:
          - {version: "5.6", skippable: true}
          - {version: "7.0", skippable: true}
          - {version: "7.1", skippable: true}
          - {version: "7.2", skippable: true}
          - {version: "7.3"}
          - {version: "7.4-rc", skippable: true}
        exclude:
          # Exclude all combination using both skippable DB and PHP versions
          - {db: {skippable: true}, php: {skippable: true}}

I tried following variant, without success.

exclude:
# Exclude all combination using both skippable DB and PHP versions
- {db: {image-name: "*", skippable: true}, php: {version: "*", skippable: true}}

It could be really usefull to be able to use wildcards (or even expressions) in exclusion rules.

Second, I tried to use include feature, but I had an an error.

matrix:
        db-image:
          - "mariadb_10.1"
          - "mariadb_10.2"
          - "mariadb_10.3"
          - "mariadb_10.4"
          - "mysql_5.6"
          - "mysql_5.7"
          - "mysql_8.0"
        php-version:
          - "5.6"
          - "7.0"
          - "7.1"
          - "7.2"
          - "7.3"
          - "7.4-rc"
        include:
          # Mark all db images as skippable except "mariadb_10.4"
          - db-image: "mariadb_10.1"
            db-image-skippable: true
          - db-image: "mariadb_10.2"
            db-image-skippable: true
          - db-image: "mariadb_10.3"
            db-image-skippable: true
          - db-image: "mysql_5.6"
            db-image-skippable: true
          - db-image: "mysql_5.7"
            db-image-skippable: true
          - db-image: "mysql_8.0"
            db-image-skippable: true
          - php-version: "5.6"
            php-version-skippable: true
          - php-version: "7.0"
            php-version-skippable: true
          - php-version: "7.1"
            php-version-skippable: true
          - php-version: "7.2"
            php-version-skippable: true
          - php-version: "7.4-rc"
            php-version-skippable: true
        exclude:
          # Exclude all combination using both skippable DB image and PHP version
          - {db-image-skippable: true, php-version-skippable: true}

Error is:

### ERRORED 08:59:06Z

- Your workflow file was invalid: .github/workflows/ci.yml (Line: 80, Col: 14): Matrix exclude key 'db-image-skippable' does not match any key within the matrix

It could be usefull too to use included variables in exclusion rules.

Regards

2 Likes

Hi @cedric-anne,

Thanks for this feedback! We’re always working to improve GitHub and the GitHub Community Forum, and we consider every suggestion we receive. I’ve logged your feature request in our internal feature request list. Though I can’t guarantee anything or share a timeline for this, I can tell you that it’s been shared with the appropriate teams for consideration.

This is even more important if you got three dimensional build array, as removing some entries often doesn’t harm, but boosts CI performance.

Also, supporting YAML anchors (https://github.community/t5/GitHub-Actions/Support-for-YAML-anchors/td-p/30336) would provide some workaround. I could split matrix into few smaller and managable ones, each per one defined job in the workflow, and re-use “steps” section with these anchors.

In case it helps anyone, I had a similar desire and got it working by “cheating” for one dimension by putting a signle entry in the matrix and additional entries in the include:

There might even be a way to simplify this further, didn’t look. Note that os seems required, otherwise they wouldn’t work