I noticed that the check_suite event that GitHub apps can subscribe to is only triggered for new code push to the repo the app is installed on. The event is not created when another user creates a fork to submit a patch to your repo.
Here is an example, where this happend:
https://github.com/jazzband/pip-tools/pull/784#issuecomment-480476249
Everything is setup following this guide:
https://developer.github.com/apps/quickstart-guides/creating-ci-tests-with-the-checks-api/
I think it't a stretch to say, that you want your CI to run for contributions from other users too.
How do I make this work for all pull requests no matter the origin, or better, can we please just sent the check_suite event for all pull requests?
The team is definitely aware of this problem and friction point, and it's something they want to improve and resolve. The problem is not trivial, so a full solution from our end will probably not happen in the near future.
For now, the recommended workaround is that in addition to listening for check_suite events -- your App also listens to pull_request events which are triggered when a pull request is updated (the synchronize event). If you receive the event for a pull request made from a fork -- you can use that to manually create check runs for the latest commit in the pull request.
I hope that helps!
For those like me who might find this and wonder how the suggested workaround affects the tutorial.
In Step 1.1 Updating app permissions, perform the following additional steps:
In Step 1.2 Add Event Handling, paste this modified version of the check_suite route so that it can also handle a pull_request event.
case request.env['HTTP_X_GITHUB_EVENT']
when 'check_suite'
# A new check_suite has been created. Create a new check run with status queued
if @payload['action'] == 'requested' || @payload['action'] == 'rerequested'
create_check_run
end
when 'pull_request'
# A new check_suite has been created. Create a new check run with status queued
if @payload['action'] == 'opened' || @payload['action'] == 'updated'
create_check_run_from_pull_request
end
end
In Step 1.3 Creating a check run, under helpers do, add this create_check_run_from_pull_request method:
# Create a new check run with the status queued def create_check_run_from_pull_request # # At the time of writing, Octokit does not support the Checks API yet, but # it does provide generic HTTP methods you can use: # https://developer.github.com/v3/checks/runs/#create-a-check-run check_run = @installation_client.post( "repos/#{@payload['repository']['full_name']}/check-runs", { # This header allows for beta access to Checks API accept: 'application/vnd.github.antiope-preview+json', # The name of your check run. name: 'Octo RuboCop', # The payload structure differs depending on whether a check run or a check suite event occurred. head_sha: @payload['pull_request']['head']['sha'] } ) end
In Troubleshooting, encourage Github to acknowledge this limitation and workaround.