Help
cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Copilot Lvl 3
Message 1 of 1

Proposal: How to test for a failed bash cmd, and mark it as a PASS only if failed?

We have logic that fails the build on purpose with a non-zero exit code.  Think, parameter validation checks that we expect to fail.

 

I would like to setup a set of workflow jobs in GitHub actions to submit invalid parameters, error, capture the error, test for a specific error code, and exit with 0 (pass) so the builds continue on.

 

More specifically, I want to use the Matrix option to pass in all possible combinations - and expect things to fail, but not fail the entire workflow and all jobs.

 

This would mean each Job would need a way to say, "If matrix.ParameterX == Y, then Expect Job to Fail normally"

 

The matrix parameters are: device, arch, boot

 

# +----------+---------+------+------+
# |  device  |  arch   | boot |  CI  |
# +----------+---------+------+------+
# | mark-one | imx53   | uSD  | PASS |
# | mark-one | imx53   | eMMC | FAIL |
# | mark-one | imx6ul  | uSD  | FAIL |
# | mark-one | imx6ul  | eMMC | FAIL |
# | mark-one | imx6ull | uSD  | FAIL |
# | mark-one | imx6ull | eMMC | FAIL |
# +----------+---------+------+------+
# | mark-two | imx53   | uSD  | FAIL |
# | mark-two | imx53   | eMMC | FAIL |
# | mark-two | imx6ul  | uSD  | PASS |
# | mark-two | imx6ul  | eMMC | PASS |
# | mark-two | imx6ull | uSD  | PASS |
# | mark-two | imx6ull | eMMC | PASS |
# +----------+---------+------+------+

As you can see, I have two different devices, 3 different arches, and 2 different boot options.

 

And as you can see, 5 will pass where 7 will error as we expect. 

 

I want to test those 7 fail conditions and pass the build (and I want to FAIL the build if one of those 7 parameters do pass!).  However, I need to test for the matrix input parameters to ensure which Jobs are expected to fail, and others to pass.

 

I would propose something like this:

 

jobs:
  build:
    runs-on: ubuntu-18.04
    strategy:
      fail-fast: true
      matrix:
        device: [mark-one, mark-two]
        arch: [imx53, imx6ul, imx6ull]
        boot: [uSD, eMMC]
    steps:
    -  run: make V=${{ matrix.device }} IMX=${{ matrix.arch }} BOOT=${{ matrix.boot }} check_version
        if: ${{ matrix.device }} == 'mark-one' \ 
              && ( ${{ matrix.arch }} != 'imx53' && ${{ matrix.boot }} != 'uSD' )
        if-match:
          allow-failure: true
        

Not the best approach. 

 

I could do some funky Bash capture and inverting, like this:

 

run: |
  make target BOOT=Invalid; \
  [[ "$?" -ne "0" ]] && echo "EXPECTED PROPER FAILURE, PASSING"
  exit 0

 

The trick being the semi-colon ";" after the first parameter, so it doesn't exit with that code and continues to the next statement of testing for the error code.

 

However, all other Steps would need "if" exceptions - and I have quite a lot.  

 

Humm, is there a way to exit the entire Job earlier and skip all remaining Steps?  I could do that instead, which would save me all of the extra "if" statements.

 

For now, I'll go this route. If someone else has suggestions, please advise.

 

---

 

Edit: Yeah, this is just insane...

 

jobs:
  build:
    runs-on: ubuntu-18.04
    strategy:
      fail-fast: false
      matrix:
        device: [mark-one, mark-two]
        arch: [imx53, imx6ul, imx6ull]
        boot: [uSD, eMMC]
    steps:
    - uses: actions/checkout@v1

    - name: Prerequisites
      run: |
        sudo apt-get update
        sudo apt-get install -y \
          bc binfmt-support bzip2 fakeroot gcc gcc-arm-linux-gnueabihf git \
          gnupg make parted qemu-user-static wget xz-utils zip debootstrap \
          sudo dirmngr bison flex libssl-dev kmod

    - name: Import Signing Keys
      run: |  # public key servers keep timing out w/parallel calls
        retry=0; until [ $retry -ge 5 ]; do
          gpg --keyserver hkp://keys.gnupg.net --recv-keys 38DBBDC86092693E 87F9F635D31D7652 && break
          n=$[$n+1]
          sleep $((1 + RANDOM % 30))
        done
    
    - run: make V=mark-one IMX=imx53 BOOT=uSD check_version
      if: matrix.device == 'mark-one' && matrix.arch == 'imx53' && matrix.boot == 'uSD'
    - run: make V=mark-one IMX=imx53 BOOT=uSD linux-deb
      if: matrix.device == 'mark-one' && matrix.arch == 'imx53' && matrix.boot == 'uSD'
    - run: make V=mark-one IMX=imx53 BOOT=uSD debian 
      if: matrix.device == 'mark-one' && matrix.arch == 'imx53' && matrix.boot == 'uSD'
    - run: make V=mark-one IMX=imx53 BOOT=uSD u-boot 
      if: matrix.device == 'mark-one' && matrix.arch == 'imx53' && matrix.boot == 'uSD'
    - run: make V=mark-one IMX=imx53 BOOT=uSD finalize
      if: matrix.device == 'mark-one' && matrix.arch == 'imx53' && matrix.boot == 'uSD'

    - run: make V=mark-two IMX=imx6ul BOOT=uSD check_version
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ul' && matrix.boot == 'uSD'
    - run: make V=mark-two IMX=imx6ul BOOT=uSD linux-deb
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ul' && matrix.boot == 'uSD'
    - run: make V=mark-two IMX=imx6ul BOOT=uSD debian 
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ul' && matrix.boot == 'uSD'
    - run: make V=mark-two IMX=imx6ul BOOT=uSD u-boot 
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ul' && matrix.boot == 'uSD'
    - run: make V=mark-two IMX=imx6ul BOOT=uSD finalize
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ul' && matrix.boot == 'uSD'

    - run: make V=mark-two IMX=imx6ul BOOT=eMMC check_version
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ul' && matrix.boot == 'eMMC'
    - run: make V=mark-two IMX=imx6ul BOOT=eMMC linux-deb
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ul' && matrix.boot == 'eMMC'
    - run: make V=mark-two IMX=imx6ul BOOT=eMMC debian 
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ul' && matrix.boot == 'eMMC'
    - run: make V=mark-two IMX=imx6ul BOOT=eMMC u-boot 
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ul' && matrix.boot == 'eMMC'
    - run: make V=mark-two IMX=imx6ul BOOT=eMMC finalize
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ul' && matrix.boot == 'eMMC'
    
    - run: make V=mark-two IMX=imx6ull BOOT=uSD check_version
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ull' && matrix.boot == 'uSD'
    - run: make V=mark-two IMX=imx6ull BOOT=uSD linux-deb
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ull' && matrix.boot == 'uSD'
    - run: make V=mark-two IMX=imx6ull BOOT=uSD debian 
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ull' && matrix.boot == 'uSD'
    - run: make V=mark-two IMX=imx6ull BOOT=uSD u-boot 
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ull' && matrix.boot == 'uSD'
    - run: make V=mark-two IMX=imx6ull BOOT=uSD finalize
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ull' && matrix.boot == 'uSD'

    - run: make V=mark-two IMX=imx6ull BOOT=eMMC check_version
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ull' && matrix.boot == 'eMMC'
    - run: make V=mark-two IMX=imx6ull BOOT=eMMC linux-deb
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ull' && matrix.boot == 'eMMC'
    - run: make V=mark-two IMX=imx6ull BOOT=eMMC debian 
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ull' && matrix.boot == 'eMMC'
    - run: make V=mark-two IMX=imx6ull BOOT=eMMC u-boot 
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ull' && matrix.boot == 'eMMC'
    - run: make V=mark-two IMX=imx6ull BOOT=eMMC finalize
      if: matrix.device == 'mark-two' && matrix.arch == 'imx6ull' && matrix.boot == 'eMMC'