Help
cancel
Showing results for 
Search instead for 
Did you mean: 
Copilot Lvl 2
Message 1 of 9

Workflow should have failed but it passed

Hello!
Im testing a workflow, it triggers fine and I wanted to make sure that a unit test would fail the workflow, so I intentionally broke one of our unit tests, running locally `yarn run test` actually fails and it even returns an exit code = 1. Committed and pushed, the workflow triggered but it passed, checked the log and the actual unit test did pass, so I believe the workflow used a cached container or container layer but I dont know how to troubleshoot that or fix it.

For reference this is my main.yml 

on: push
jobs:
  unit-test:
    runs-on: ubuntu-18.04
    steps:
    - uses: actions/checkout@v2
    - name: Use Node.js
      uses: actions/setup-node@v1
      with:
        node-version: '12.x'
    - name: Install dependencies
      run: yarn
    - name: Run unit tests
      run: yarn run test
8 Replies
Highlighted
Copilot Lvl 3
Message 2 of 9

Re: Workflow should have failed but it passed

Do you have a link to the repo and/or the log? If the actual exit code indeed is non-zero the action should have failed. To troubleshoot some stuff you could print the exit code manually. E.g. the following.

    - name: Run unit tests
      run: |
        yarn run test
        echo "Exited with '$?'"
Highlighted
GitHub Partner
Message 3 of 9

Re: Workflow should have failed but it passed

Your 'yarn run test' command does not return non-zero exit code actually. Checke on my side, 'yarn run test --watchAll=false ' command failed my workflow. Please add command echo "Exited with '$?'"  to check exit code.

yarn test failed.png

Highlighted
Copilot Lvl 2
Message 4 of 9

Re: Workflow should have failed but it passed

Thank you guys! So yeah, yarn was not exiting with status code `1` as it does in my local machine, but the actual problem is that the code that is being checked out is not the one I commited, In Actions I clicked on the commit sha to check the file that should have failed the test:

Screenshot from 2020-03-27 09-57-05.png

both of these commits have the unit test that fails locally, the code looks the same and so it should fail, but checking the output from the action that specific test passes... but it shoulnt! 

My guess is that the Action is using some cache, I belive it uses Docker under the hud so maybe an old layer is being used? 

Highlighted
Copilot Lvl 3
Message 5 of 9

Re: Workflow should have failed but it passed

Do you have a link to the repository or a minimal example repository that shows this behaviour?

Highlighted
Copilot Lvl 2
Message 6 of 9

Re: Workflow should have failed but it passed

Im afraid its a company private repo, but I dont think I've done anything out of the ordinary, I even copy pasted the yaml file minus the `yarn run test` and echo $?
:( 

Highlighted
GitHub Partner
Message 7 of 9

Re: Workflow should have failed but it passed

Hi @purefan ,

 

No, workflow will start on pure clean github hosted runner, no cache according to your workflow code.

 

but the actual problem is that the code that is being checked out is not the one I commited.

>> I'm a little confused, if you commit in other branch, why the workflow is triggered? Did you also have a workflow yaml on that branch? Please make sure you checkout the correct code.

 

The main reason is that non-zero exit code returned to github runner. step will pass if zero exit code returned.

 

Thanks.

 

Highlighted
Copilot Lvl 2
Message 8 of 9

Re: Workflow should have failed but it passed

> I'm a little confused, if you commit in other branch, why the workflow is triggered? Did you also have a workflow yaml on that branch? Please make sure you checkout the correct code.

Im confused as well, I have only commited to one branch, and commiting here triggers the Github Action (as it should because of the simple `on: push` directive 

Highlighted
GitHub Partner
Message 9 of 9

Re: Workflow should have failed but it passed

Hi @purefan ,

 

Which branch do you have the workflow yaml? If it's 'dev' branch, please make sure you commit fail-test code to 'dev' branch, not other branches.

And what's the exit code of your yarn test in workflow run(echo "Exited with '$?'")? Should be 'zero pass, non-zero fail'. 

Please sync code between local and remote, if the test result is still different, please check the env setting as well and ensure they are same. 

If it persists for you, it's recommened to share a sample repo for further investigation.

 

Thanks.