Allow nesting matrix includes

Imagine you have this job:

jobs:
test:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: ubuntu-18.04
include:
-db: postgres  
 db\_version: 9.6  
 - db: postgres  
 db\_version: 10
-db: mysql  
 db\_version: 5.6  
 - db: mysql  
 db\_version: 7

It would be really handy to be able to use lists inside the include:

include:
          - db: postgres
            db_version: [9.6, 10]
          - db: mysql
            db_version: [5.6, 7]

If you’re testing against multiple databases then this is a common pattern that the matrix doesn’t support: you want to define different database versions, but not the whole matrix of versions * vendors.

The alternative would be a huge duplicated matrix of includes for each item, or duplicated jobs with the matrix subset you want.

5 Likes

+1

I myself and trying to use a complicated matrix build; but, having to add a whole bunch of ugly if statements to get around it.

This is what i want:

name: Build-All
on: [push]

# We are using Matrix build here on Github Actions for parallel execution.
# We have 3 parameters: device, arch, boot
# While the Mark II allows for 4 builds (imx6ull, imx6ul) this matrix becomes
# an issue with the Mark One because the Makefile errors if you pass eMMC as
# the boot parameter or imx6ull/imx6ul as the arch.
#
# +----------+---------+------+------+
# | device | arch | boot | CI |
# +----------+---------+------+------+
# | mark-one | imx53 | uSD | PASS |
# | mark-one | imx53 | eMMC | FAIL |
# | mark-one | imx6ul | uSD | FAIL |
# | mark-one | imx6ul | eMMC | FAIL |
# | mark-one | imx6ull | uSD | FAIL |
# | mark-one | imx6ull | eMMC | FAIL |
# +----------+---------+------+------+
# | mark-two | imx53 | uSD | FAIL |
# | mark-two | imx53 | eMMC | FAIL |
# | mark-two | imx6ul | uSD | PASS |
# | mark-two | imx6ul | eMMC | PASS |
# | mark-two | imx6ull | uSD | PASS |
# | mark-two | imx6ull | eMMC | PASS |
# +----------+---------+------+------+
#
# Therefore, there are really only 5 successful builds, and 7 expected failed
# builds per the above matrix. However, Github Actions doesn't have a clean
# way of testing for failed builds with specific matrix params.
#

jobs:
  build:
    runs-on: ubuntu-18.04
    strategy:
      fail-fast: false
      matrix:
        device: [mark-one, mark-two]
        arch: [imx53, imx6ul, imx6ull]
        boot: [uSD, eMMC]
    if: |
      (matrix.device == 'mark-one' && matrix.arch == 'imx53' && matrix.boot == 'uSD') \
      || (matrix.device == 'mark-two' && matrix.arch == 'imx6ul' && matrix.boot == 'uSD') \
      || (matrix.device == 'mark-two' && matrix.arch == 'imx6ul' && matrix.boot == 'eMMC') \
      || (matrix.device == 'mark-two' && matrix.arch == 'imx6ull' && matrix.boot == 'uSD') \
      || (matrix.device == 'mark-two' && matrix.arch == 'imx6ull' && matrix.boot == 'eMMC')
    steps:
    - uses: actions/checkout@v1

    - name: Prerequisites
      run: |
        sudo apt-get update
        sudo apt-get install -y \
          bc binfmt-support bzip2 fakeroot gcc gcc-arm-linux-gnueabihf git \
          gnupg make parted qemu-user-static wget xz-utils zip debootstrap \
          sudo dirmngr bison flex libssl-dev kmod

    - name: Import Signing Keys
      run: | # public key servers keep timing out w/parallel calls
        retry=0; until [$retry -ge 5]; do
          gpg --keyserver hkp://keys.gnupg.net --recv-keys 38DBBDC86092693E 87F9F635D31D7652 && break
          n=$[$n+1]
          sleep $((1 + RANDOM % 30))
        done

    - run: make V=${{ matrix.device }} IMX=${{ matrix.arch }} BOOT=${{ matrix.boot }} check_version
    - run: make V=${{ matrix.device }} IMX=${{ matrix.arch }} BOOT=${{ matrix.boot }} linux-deb
    - run: make V=${{ matrix.device }} IMX=${{ matrix.arch }} BOOT=${{ matrix.boot }} debian
    - run: make V=${{ matrix.device }} IMX=${{ matrix.arch }} BOOT=${{ matrix.boot }} u-boot
    - run: make V=${{ matrix.device }} IMX=${{ matrix.arch }} BOOT=${{ matrix.boot }} finalize

This is what I have to do to get around it today (oh so ugly in the Actions panel):

name: Build-All
on: [push]

# We are using Matrix build here on Github Actions for parallel execution.
# We have 3 parameters: device, arch, boot
# While the Mark II allows for 4 builds (imx6ull, imx6ul) this matrix becomes
# an issue with the Mark One because the Makefile errors if you pass eMMC as
# the boot parameter or imx6ull/imx6ul as the arch.
#
# +----------+---------+------+------+
# | device | arch | boot | CI |
# +----------+---------+------+------+
# | mark-one | imx53 | uSD | PASS |
# | mark-one | imx53 | eMMC | FAIL |
# | mark-one | imx6ul | uSD | FAIL |
# | mark-one | imx6ul | eMMC | FAIL |
# | mark-one | imx6ull | uSD | FAIL |
# | mark-one | imx6ull | eMMC | FAIL |
# +----------+---------+------+------+
# | mark-two | imx53 | uSD | FAIL |
# | mark-two | imx53 | eMMC | FAIL |
# | mark-two | imx6ul | uSD | PASS |
# | mark-two | imx6ul | eMMC | PASS |
# | mark-two | imx6ull | uSD | PASS |
# | mark-two | imx6ull | eMMC | PASS |
# +----------+---------+------+------+
#
# Therefore, there are really only 5 successful builds, and 7 expected failed
# builds per the above matrix. However, Github Actions doesn't have a clean
# way of testing for failed builds with specific matrix params.
#

jobs:
  build:
    runs-on: ubuntu-18.04
    strategy:
      fail-fast: false
      matrix:
        device: [mark-one, mark-two]
        arch: [imx53, imx6ul, imx6ull]
        boot: [uSD, eMMC]
    steps:
    - uses: actions/checkout@v1

    - name: Prerequisites
      run: |
        sudo apt-get update
        sudo apt-get install -y \
          bc binfmt-support bzip2 fakeroot gcc gcc-arm-linux-gnueabihf git \
          gnupg make parted qemu-user-static wget xz-utils zip debootstrap \
          sudo dirmngr bison flex libssl-dev kmod

    - name: Import Signing Keys
      run: | # public key servers keep timing out w/parallel calls
        retry=0; until [$retry -ge 5]; do
          gpg --keyserver hkp://keys.gnupg.net --recv-keys 38DBBDC86092693E 87F9F635D31D7652 && break
          n=$[$n+1]
          sleep $((1 + RANDOM % 30))
        done

    - run: make V=mark-one IMX=imx53 BOOT=uSD check_version
      if: matrix.device == 'mark-one' && matrix.arch == 'imx53' && matrix.boot == 'uSD'
    - run: make V=mark-one IMX=imx53 BOOT=uSD linux-deb
      if: matrix.device == 'mark-one' && matrix.arch == 'imx53' && matrix.boot == 'uSD'
    - run: make V=mark-one IMX=imx53 BOOT=uSD debian
      if: matrix.device == 'mark-one' && matrix.arch == 'imx53' && matrix.boot == 'uSD'
    - run: make V=mark-one IMX=imx53 BOOT=uSD u-boot
      if: matrix.device == 'mark-one' && matrix.arch == 'imx53' && matrix.boot == 'uSD'
    - run: make V=mark-one IMX=imx53 BOOT=uSD finalize
      if: matrix.device == 'mark-one' && matrix.arch == 'imx53' && matrix.boot == 'uSD'

    - run: make V=mark-two IMX=imx6ul BOOT=uSD check_version
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ul' && matrix.boot == 'uSD'
    - run: make V=mark-two IMX=imx6ul BOOT=uSD linux-deb
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ul' && matrix.boot == 'uSD'
    - run: make V=mark-two IMX=imx6ul BOOT=uSD debian
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ul' && matrix.boot == 'uSD'
    - run: make V=mark-two IMX=imx6ul BOOT=uSD u-boot
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ul' && matrix.boot == 'uSD'
    - run: make V=mark-two IMX=imx6ul BOOT=uSD finalize
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ul' && matrix.boot == 'uSD'

    - run: make V=mark-two IMX=imx6ul BOOT=eMMC check_version
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ul' && matrix.boot == 'eMMC'
    - run: make V=mark-two IMX=imx6ul BOOT=eMMC linux-deb
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ul' && matrix.boot == 'eMMC'
    - run: make V=mark-two IMX=imx6ul BOOT=eMMC debian
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ul' && matrix.boot == 'eMMC'
    - run: make V=mark-two IMX=imx6ul BOOT=eMMC u-boot
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ul' && matrix.boot == 'eMMC'
    - run: make V=mark-two IMX=imx6ul BOOT=eMMC finalize
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ul' && matrix.boot == 'eMMC'

    - run: make V=mark-two IMX=imx6ull BOOT=uSD check_version
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ull' && matrix.boot == 'uSD'
    - run: make V=mark-two IMX=imx6ull BOOT=uSD linux-deb
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ull' && matrix.boot == 'uSD'
    - run: make V=mark-two IMX=imx6ull BOOT=uSD debian
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ull' && matrix.boot == 'uSD'
    - run: make V=mark-two IMX=imx6ull BOOT=uSD u-boot
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ull' && matrix.boot == 'uSD'
    - run: make V=mark-two IMX=imx6ull BOOT=uSD finalize
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ull' && matrix.boot == 'uSD'

    - run: make V=mark-two IMX=imx6ull BOOT=eMMC check_version
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ull' && matrix.boot == 'eMMC'
    - run: make V=mark-two IMX=imx6ull BOOT=eMMC linux-deb
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ull' && matrix.boot == 'eMMC'
    - run: make V=mark-two IMX=imx6ull BOOT=eMMC debian
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ull' && matrix.boot == 'eMMC'
    - run: make V=mark-two IMX=imx6ull BOOT=eMMC u-boot
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ull' && matrix.boot == 'eMMC'
    - run: make V=mark-two IMX=imx6ull BOOT=eMMC finalize
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ull' && matrix.boot == 'eMMC'
2 Likes

@orf - Thanks for sharing your use case here! We’ve noted this internally for our product team’s consideration, though we can’t make any promises on specific changes.

If you have any additional product feedback or feature requests, we suggest writing in to our new contact form – that’s where our team is collective product feedback for GitHub Actions and other GitHub features:

https://support.github.com/contact/feedback?contact%5Bcategory%5D=actions

Hello, I just made another thread on this same issue (shame) - Feature request and use case example to allow `matrix` in `if`s.

I hope this additional use case example will help. I’ll duplicate this via the feedback form.