Semantically incorrect workflows are not listed as failed in PRs

Example: we have a workflow with job like this:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - run: echo $FOO
        env:
          FOO: A
          BAR: B

Then we have a PR adding another line with key FOO, so the resulting env: looks like this:

FOO: A
          BAR: B
          FOO: C

Such yaml is technically correct (though I believe it’s behavior is undefined, since some parsers emit FOO being A, and other being C), but GHA will argue that the FOO key is duplicated. However said workflow won’t be marked as failed in Pull Request interface, and such PR can be silently merged without any indication that something went wrong. See https://github.com/Artalus/workflow-pr-failing/pull/1

@artalus ,

Yeah, I can reproduce this question.

I think the following points may explain the question:

1) When a workflow is triggered, GitHub will verify the workflow YAML file at first to see if there are some wrongs (syntax, context format, etc.). If the workflow YAML file has wrongs, the workflow run will not be started up.

2) In the PR’s checks list, it only lists the jobs in the started up workflow runs. If a workflow run is not started up, its jobs will not be listed.

If you want the jobs in the not started up workflow can be listed in the PR’s checks list, as a workaround, you can set a Branch protection rule to enable the “Require status checks to pass before merging” option, and mark these jobs as " Required".

For example:

It’s a useful info, but it does not fix the problem.

https://github.com/Artalus/workflow-pr-failing/pull/2

Here I have a workflow that triggers only when a specified file changes. Setting repo to wait for this workflow will block all PRs that do not change the file, since the said workflow will never start for such PRs.