GitHub actions are severely limited on PRs

Hi there,

I think GitHub Actions are ruined by a very tiny detail which should be fixed ASAP. More specifically, GitHub actions running on PRs are so severerly limited they often become useless.

On forks and PRs

I think everyone will agree that PRs from forks are pretty much the de facto standard way of collaborating on GitHub. For instance, let’s say you want to contribute to Ruby on Rails. The recommended way is to :

On Actions

Now, what’s the point of actions? Well, they can solve some of those problems:

  • Running some checks on the code base (for instance, a linter)
  • Running a test suite
  • Getting some metrics about the code base (e.g. sloc count)
  • Benchmarking the code (finding out how fast the code goes)
  • Measuring the products (what’s the size of the binary once built, or the size of the JS once mifinied/gzipped)

As you can see, some problems answer a boolean question : for instance the test suite either passes or not. That can be the case of a linter too, which either has some work to do or doesn’t need to. For those problems, actions are fine, they work just great.

Now there are other problems which do not have a boolean answer. For instance a measuring the speed of the code or the memory usage of a library isn’t a yes/no kind of question. Those problems simply cannot be currently solved by actions.

The problem

Funnily enough, the problem is not answering the question (e.g. how fast is the code), it’s about reporting the answer. Indeed, there are two ways to output something from a GitHub action:

* First, using a return code. If you return a non-zero code, then the action is considered a failure. Otherwise, it’s a pass. That works great for boolean problems!

* Otherwise, you can use the GitHub API (using a provided GITHUB_TOKEN) to do pretty much whatever you want (create a new issue, write a comment, add a label, request a review, etc…).

However, for security reasons, the team at GitHub has decided to issue a GITHUB_TOKEN that has read-only access to the entire GitHub repository when the action is run from a fork.

In other words, when you create a pull request on Ruby on Rails, any action running on your PR cannot do anything at all on the Ruby on Rails repository. That includes doing anything at all on your very own PR. I repeat: the only output the actions running on your PR can have are “fail/pass”. That’s it. Of course it’s a good idea to limit what those actions could do, but at the moment they simply cannot do anything at all apart from returning a pass/fail status. Namely, they cannot comment on your PR, they cannot add labels to your PR, etc…

That’s a really bad problem, that’s been reported several times, and nothing has been done about this. It badly limits the usefulness of GitHub actions. By the way, “HttpError: Resource not accessible by integration” is the message you’ll get when you’ll be bitten by this, as an action developer.

Some possible solutions

I can imagine two solutions to this:

  • Provide actions running on PR with a GitHub token that can act on the PR itself only.
  • Allow Actions to return more than a “fail/pass” result. Maybe at least a string that would be commented on the PR would be a good start?

Links

38 Likes

Currently, the GITHUB_TOKEN only has read permission to pull requests when access by forked repos. 

This is refer to official document: https://help.github.com/en/actions/configuring-and-managing-workflows/authenticating-with-the-github_token#permissions-for-the-github_token 

For all your requirements on PR from forked repo, I would encourage you sharing those in the Feedback form for GitHub Actions. Thank you for your understanding. 

1 Like

Seriously? A canned answer? I obviously did read the documentation. You’ve missed the entire point. That’s shameful.

4 Likes

Hi @ecco , 

I really understand your scenario and you are looking forward to a write permission of GITHUB_TOKEN to act on pull request from fork.  

But I am so sorry that this is not supported now. 

Feedback form for GitHub Actions is a more suitable site to share your idea : adding some additional scopes to the GITHUB_TOKEN to enable commenting , labeling on PRs. The feedback will be checked by GitHub Engineering team directly. 

>>Allow Actions to return more than a “fail/pass” result. Maybe at least a string that would be commented on the PR would be a good start?

There is a set-output command which could set output variable for a step with string value.  You could use the output variable in the next steps of current job. https://github.com/actions/toolkit/blob/master/docs/commands.md#set-outputs 

But unfortunately,  it still could not be commented on the PR.  

3 Likes

May this be a working workaround for this problem?

https://github.com/nyurik/auto_pr_comments_from_forks

1 Like

OK, here’s an experiment I made where I was sure it would work and be a workaround for the problem you describe. But I was wrong…

First, create an access token in your settings with sufficient “repo” permissions on the repo you have set up your actions.

Store that token as secret, e.g. RW_TOKEN, in the repo you have set up your actions.

Create an action YML:

name: create issue on PR/push

on: [pull_request,push]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - name: Run a multi-line script
      env:
        RW_TOKEN: ${{ secrets.RW_TOKEN }}
      shell: bash
      run: |
        curl --request POST \
        --url https://api.github.com/repos/myaccount/mytestrepo/issues \
        --header 'authorization: Bearer'" $RW_TOKEN" \
        --header 'content-type: application/json' \
        --data '{
          "title": "Automated issue for commit: ${{ github.sha }}",
          "body": "This issue was automatically created by the GitHub Action workflow **${{ github.workflow }}**. \n\n The commit hash was: _${{ github.sha }}_."
          }'

This code, using the RW_TOKEN copied and run in a terminal on my machine executes just fine. Triggered by a push, the action runs just fine and create the issue. Triggered by a PR from a forked repo , it fails with:

{
  "message": "Bad credentials",
  "documentation_url": "https://developer.github.com/v3"
}

THIS IS JUST CRAZY, isn’t it??? 

1 Like

As frustrating as this is, it does make sense that the same issue would apply to secrets as it does for the built in GitHub Token. Otherwise that would be a major security flaw. However this is a MAJOR oversight on GitHub’s part. There needs to be some solution to handle this extremely common workflow.

3 Likes

This actually makes a majority of use cases that I have for GitHub Actions useless and would force me to look to external integrations. Can someone from GitHub comment on whether this is being investigated? Or is this “just the way things are going to be”?

1 Like

I have seen mention elsewhere that this is (was) being looked into. I’ll see if I can dig the details up again.

1 Like

Agreed - the #1 use for github actions in my mind is around making the contribution process simpler, automated, and bullet proof. But since we can’t support any features that matter with contributed updates then it is useless.

All we need is the ability to automatically use the workflow from the TARGET branch instead of the PR itself – but giving it access to the branch being merged. It really doesn’t seem that difficult? Although I am sure I am missing something?

2 Likes

I’ve updated the add-pr-comment action I maintain to make this as easy as possible until GitHub comes up with a better solution. I created a simple proxy app which can be deployed via a Cloud Run Button, and allows specifying the proxy URL as an override in the action. I imagine most use will fall in the Cloud Run free tier.

2 Likes

I accomplished this without a github app actually. @mshick this actually maeks PR comments on forked PR as well as run full checks / linting on them so it may be a good example of how you could accomplish your action as yours would be 10,000x simpler :slight_smile:

1 Like

So the artifacts generated from the fork are available to the scheduled run on the upstream / original repo? Great discovery, thanks!

I’ll look at incorporating something similar, could definitely be useful for those who don’t mind the lag time and would rather not host a proxy.

2 Likes

Only when using the REST API - the @actions/artifact specifically restricts it to the current workflow run only and doesn’t make it available to the workflow as a whole.

I was planning at some point to bundle it up into a easy workflow to use, if possible.

1 Like

GitHub has introduced a new event type: pull_request_target, which allows to run workflows from base branch and pass a token with write permission.

In order to solve this, we’ve added a new pull_request_target event, which behaves in an almost identical way to the pull_request event with the same set of filters and payload. However, instead of running against the workflow and code from the merge commit, the event runs against the workflow and code from the base of the pull request. This means the workflow is running from a trusted source and is given access to a read/write token as well as secrets enabling the maintainer to safely comment on or label a pull request. This event can be used in combination with the private repository settings as well.

3 Likes

I believe I tried that, but since it runs against the workflow and code from the base of the pull request, it makes it impossible for my action to retrieve the list of changes files from the forked repo to run tests on them. The alternative (in my case) would be to re-rest the entire repo, not only changed files.

I’d loved to be proven wrong here, so if anyone knows, feel free to comment.

1 Like

I think pull_request_target, can do what you need. Just make sure your checkout gets what you need, ex:

     uses: actions/checkout@v2
      with:
        ref: ${{github.event.pull_request.head.ref}}
        repository: ${{github.event.pull_request.head.repo.full_name}}

For changed/added files you can also checkout: futuratrepadeira/changed-files.

Here’s an example that you could use as a model:

2 Likes

Is this still safe if the action calls any scripts from the repo? If the GITHUB_TOKEN is not stored in an env var I don’t think a malicious PR could steal the token…

1 Like

GITHUB_TOKEN is fairly safe as it is only valid for the duration of the workflow run.

1 Like

I disagree with this; if the token was in an env var (it isn’t, I tested it), then any CI where the workflow yaml calls external scripts in the repo would be vulnerable to someone sending a PR that modifies such a script in order to abuse the token.

Sure, the impact would be limited, but you could still use it to cause some chaos in the repo! And I could totally imagine script kiddies searching GitHub for repos vulnerable to this programmatically “just for the lulz”.

1 Like