Help
cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Ground Controller Lvl 1
Message 1 of 4

GraphQL GitHub API: How to get more than 1000 Pull Requests?

Hi,

 

I'm unable to get more than 1000 PRs from the GraphQL API. I'm aware of the resource limitations, but 1000 nodes is far from the limit of 50000 stated in that document. I think I'm incorrectly using pagination, but can't figure out where exactly I'm wrong here.

 

Can anybody help me to understand the reason?

 

Here is a simplified version of my code:

#!/usr/bin/env python

from __future__ import print_function

import os
import requests
import sys
import time

QUERY = """
{
  search(query: "is:pr repo:kubernetes/kubernetes created:>2016-01-01", type: ISSUE, first: 100, %s) {
    issueCount
    pageInfo {
        endCursor
        hasNextPage
    }
    edges {
      cursor
      node {
        ... on PullRequest {
          number
        }
      }
    }
  }
}
"""

def main(argv):
    if len(argv) > 1:
        token = argv[1]
    else:
        token = os.getenv('GITHUB_TOKEN')
    if not token:
        print("Usage: prs <github token>")
        return 1

    after = ''
    while True:
        request = requests.post('https://api.github.com/graphql',
                                json={'query': QUERY % after},
                                headers={"Authorization": "Bearer %s" % token})
        result = request.json()

        if 'errors' in result:
            print(result)
            return 1

        search = result['data']['search']

        print("Issue count:", search['issueCount'], "number of edges:", len(search['edges']))

        print("PageInfo:", search['pageInfo'], "cursor:", search['edges'][-1]['cursor'], "\n")

        if not search['pageInfo']['hasNextPage']:
            print("done")
            break

        after = 'after: "%s"' % search['edges'][-1]['cursor']
        time.sleep(1)

if __name__ == '__main__':
    sys.exit(main(sys.argv))

and its output:

 

$ ./prs-test 
Issue count: 33754 number of edges: 100
PageInfo: {u'endCursor': u'Y3Vyc29yOjEwMA==', u'hasNextPage': True} cursor: Y3Vyc29yOjEwMA== 

Issue count: 33754 number of edges: 100
PageInfo: {u'endCursor': u'Y3Vyc29yOjIwMA==', u'hasNextPage': True} cursor: Y3Vyc29yOjIwMA== 

Issue count: 33754 number of edges: 100
PageInfo: {u'endCursor': u'Y3Vyc29yOjMwMA==', u'hasNextPage': True} cursor: Y3Vyc29yOjMwMA== 

Issue count: 33754 number of edges: 100
PageInfo: {u'endCursor': u'Y3Vyc29yOjQwMA==', u'hasNextPage': True} cursor: Y3Vyc29yOjQwMA== 

Issue count: 33754 number of edges: 100
PageInfo: {u'endCursor': u'Y3Vyc29yOjUwMA==', u'hasNextPage': True} cursor: Y3Vyc29yOjUwMA== 

Issue count: 33754 number of edges: 100
PageInfo: {u'endCursor': u'Y3Vyc29yOjYwMA==', u'hasNextPage': True} cursor: Y3Vyc29yOjYwMA== 

Issue count: 33754 number of edges: 100
PageInfo: {u'endCursor': u'Y3Vyc29yOjcwMA==', u'hasNextPage': True} cursor: Y3Vyc29yOjcwMA== 

Issue count: 33754 number of edges: 100
PageInfo: {u'endCursor': u'Y3Vyc29yOjgwMA==', u'hasNextPage': True} cursor: Y3Vyc29yOjgwMA== 

Issue count: 33754 number of edges: 100
PageInfo: {u'endCursor': u'Y3Vyc29yOjkwMA==', u'hasNextPage': True} cursor: Y3Vyc29yOjkwMA== 

Issue count: 33754 number of edges: 100
PageInfo: {u'endCursor': u'Y3Vyc29yOjEwMDA=', u'hasNextPage': False} cursor: Y3Vyc29yOjEwMDA= 

done

Thank you,

Ed

3 Replies
Community Manager
Message 2 of 4

Re: GraphQL GitHub API: How to get more than 1000 Pull Requests?

Hi Ed,

 

Apologies for the delay response, are you still having issues with pagination?

Thanks for being here, sharing, and keeping our community awesome!
Best,
AndreaG

Mark helpful posts with Accept as Solution to help other users locate important info. Don't forget to give Kudos for great content!

Copilot Lvl 2
Message 3 of 4

Re: GraphQL GitHub API: How to get more than 1000 Pull Requests?

I am facing the same issue.

It seems the node in position 1000 just doesnt have a next cursor to go to.

 

There is very little documentation or support for this problem.

 

please assist.

Ground Controller Lvl 1
Message 4 of 4

Re: GraphQL GitHub API: How to get more than 1000 Pull Requests?

I'm facing too.

This limitation should be documented.