Currently, using 'continue-on-error:' in a step flags the step with a green check and allows the job to continue. The only way to see if the step failed is to look at the step's log.
Other CI systems allow the UI indication of the failure to propagate up to the job level, so it's easy to see if the there was an 'allowed failure'.
Since 'continue-on-error:' is step scoped, it would be helpful to have something similar to 'allow-failure' at the job level, and have the job level UI indication reflect whether there was a failure.
I suspect normal use would use an expression based on matrix values.
A common use of 'allow-failure' is to test 'packages' against their respective language's master branch.
Instead of using `continue-on-error`, you may want to execute subsequent steps even if there was a failure.
This isn't on the job level, but will preserve the failure of a particular step but allow subsequent steps to complete. You can do this with conditionals.
steps: - run: false - run: echo runs on success or failure if: succeeded() || failed() - run: echo runs always, even if the prior step was cancelled if: always()
Thanks for the response. Apologies, I wasn't clear.
Here's a build on Travis:
It has 'Allowed Failures` jobs shown, and some failed. The commit that the build is associated with shows a green check. So, we're one or two UI clicks away from determining whether any 'Allowed Failures` jobs actually failed.
I don't know what the interface is between a CI provider and the UI that displays whether a commit or PR passed the CI, but, at present, determining the status of an 'Allowed Failures` job with Actions is a PITA.
I would like to +1 this; this would be convenient for upgrade paths that should not break the build but give a warning. For example, when you are working towards improving your coding standards or testing against experimental branches of libraries or languages.
I'd also like to point out that this is very needed. :)
I'm testing some experimental functionality, and I don't want it to hold up my build, but I do want it part of my CI, so that I can keep the experiment up to date
If I understand correctly, another use for this would be to flag linting errors. In general I want all PRs to come with nice clean linting reports, but I don't want to fail the build and prevent artifacts from uploading just because someone missed a single white space somewhere.
I don't know, I'm relatively new to workflows so maybe I missed another way to do this. But it's important to understand that there is a difference between an error and a warning. Errors do mean something has broken, warnings do not necessarily show something has broken, but should be presented somewhere obvious so they can be assessed.
You'd actually see an entry in the "Annotations" snippet on the bottom right of the page, but unfortunately the step is still marked with the green check mark for success, instead of something indicating "allowed failure". Also, the annotation entry links to .github:1, which is of course unhelpful for finding where the error has occurred.
Hello, in a suite of job variants per matrix variable combinations, this would indeed by nice, to test the tester.
On the one hand, if a workflow fails, all the matrix combinations are blocked. Some options could allow all of the other combinations to keep running.
The continue-on-error option puts a green tick to a step, it could some orange tick instead...
Related non-github-internal project-specific issue: https://github.com/douglaskastle/blender-addon-tester/issues/16#issuecomment-605697324