@anderspitman When a workflow is triggered by pull request event, the github.ref will be refs/pull/<id>/merge. And the github.sha will be the pre merge commit id. If you have a pull request from dev to master, github.sha is not the commit which you pushed to dev.  

pr event.png768×699 29.5 KB

You need to use github.event.pull_request.head.sha to get the commit id which you pushed to dev. 

And based on my test when the workflow is triggered by push to master,  comment PR step will fail. 

So you could change your workflow file to 

name: CI
on:
  pull_request:
    branches: [master]
jobs:

example:

name: sample comment

runs-on: ubuntu-latest

steps:

- name: comment PR

uses: unsplash/comment-on-pr@master

env:

GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

with:

msg: "example.com/${{github.event.pull_request.head.sha}}"