Run next step even though previous fails

Hey guys,

I looked in the forum before and the documentation, but all ways I tried seems not to work with our workflow. So I’m sorry if I missed something and this is a duplication.

We have two packages in our python project which we want to test individually. Currently a job fails when the first test suite fails, but I’m trying to run the second one if the first one fails, but it should not run if some of the previous fails. This is a excerpt of the workflow file.

- name: Test BrainAcademy
   id: test_brain
   run: python -m unittest discover tests_brain_academy
- name: Test Deep RL
  if: ${{ success() || steps.test_brain.outcome == failure() }}
  run: python -m unittest discover tests_deep_rl

Thank you very much for you help. I’m happy to provide more information if needed.

Marcel

Just make 

python -m ...

exit with non-zero code, like:

https://stackoverflow.com/questions/24972098/unit-test-script-returns-exit-code-0-even-if-tests-fail

you can verify this result with:

python -m ...
echo $?

Hey RammusXu, thanks for your reply. I’m not entirely sure what you meant, but I would like to change my python code just to satisfy the CI runner. The first test suite exists with 1.

Run python -m unittest discover tests_brain_academy
  python -m unittest discover tests_brain_academy
  shell: /bin/bash -e {0}
  env:
    pythonLocation: /opt/hostedtoolcache/Python/3.6.10/x64
INFO:matplotlib.font_manager:generated new fontManager
ssssss....F........sssssssssssssssssssss..................................
======================================================================
pygame 1.9.6
FAIL: test_failure (unit_tests.test_academy.TestAcademy)
Hello from the pygame community. https://www.pygame.org/contribute.html
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/BrainAcademy/BrainAcademy/tests_brain_academy/unit_tests/test_academy.py", line 20, in test_failure
    self.assertEqual(True, False)
AssertionError: True != False

----------------------------------------------------------------------
Ran 74 tests in 0.202s

FAILED (failures=1, skipped=27)
##[error]Process completed with exit code 1.

Hey RammusXu thanks for your reply. I’m not entirely sure what you meant, but I would prefer not to do this in python. And I think the first test suite exists with a non-zero.

Run python -m unittest discover tests_brain_academy
  python -m unittest discover tests_brain_academy
  shell: /bin/bash -e {0}
  env:
    pythonLocation: /opt/hostedtoolcache/Python/3.6.10/x64
INFO:matplotlib.font_manager:generated new fontManager
ssssss....F........sssssssssssssssssssss..................................
======================================================================
pygame 1.9.6
FAIL: test_failure (unit_tests.test_academy.TestAcademy)
Hello from the pygame community. https://www.pygame.org/contribute.html
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/BrainAcademy/BrainAcademy/tests_brain_academy/unit_tests/test_academy.py", line 20, in test_failure
    self.assertEqual(True, False)
AssertionError: True != False

----------------------------------------------------------------------
Ran 74 tests in 0.202s

FAILED (failures=1, skipped=27)
##[error]Process completed with exit code 1.

PS: I post this again, because the previous reply is not showing. So I hope this will not show up twice.

Thanks for your reply, but I’m not entirely sure this is wise, because I don’t want to handle this in python. In addition to that I think the first test suite exists with a non-zero code already:

Run python -m unittest discover tests_brain_academy
  python -m unittest discover tests_brain_academy
  shell: /bin/bash -e {0}
  env:
    pythonLocation: /opt/hostedtoolcache/Python/3.6.10/x64
INFO:matplotlib.font_manager:generated new fontManager
ssssss....F........sssssssssssssssssssss..................................
======================================================================
pygame 1.9.6
FAIL: test_failure (unit_tests.test_academy.TestAcademy)
Hello from the pygame community. https://www.pygame.org/contribute.html
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/BrainAcademy/BrainAcademy/tests_brain_academy/unit_tests/test_academy.py", line 20, in test_failure
    self.assertEqual(True, False)
AssertionError: True != False

----------------------------------------------------------------------
Ran 74 tests in 0.202s

FAILED (failures=1, skipped=27)
##[error]Process completed with exit code 1.

PS: I have trouble send my replies, I hope they will not show multiple times.

Hi @segelzwerg , 

You could use next code in the if conditional in “Test Deep RL” step: 

if: "success() || steps.test_brain.outcome == 'failure' "

When you use expressions in an if conditional, you may omit the expression syntax (${{ }}) because GitHub automatically evaluates the if conditional as an expression.

And possible values of step outcome are success, failure, cancelled, or skipped. failure() is a function which returns true when any previous step of a job fails.  So just use ‘failure’ instead of ‘failure()’ could work for you.  

1 Like

@yanjingzhu yes that did the trick. Thanks a lot, I didn’t use ’ previously.

Thank you very much