top 10 javascript/opensource repositories ranked by star #24415

Nanoboss · 2019-05-22T07:32:02Z

Nanoboss
May 22, 2019

I would like to get the top 10 javascript/opensource repositories ranked by star (and some related informations) using GitHub GraphQL Api in a python project. I have this query so far:

query{
  search(type: REPOSITORY, query: "language:javascript", first:10) {
    userCount
    edges {
      node {
        ... on Repository {
          name
          url
          stargazers {
            totalCount
          }
          owner{
            login
          }
        }
      }
    }
  }
}

The problem is that it does not always return the same result: it will return 10 random repositories ordered by starcount at each query rather than the absolute top 10.

And on top of that I’d like to get the ones that are open source.

I use the query

query{
    licenses{name}
}

to get a list of licences but I don’t know if this is an exhaustive list (seems like it’s missing some licenses like MIT). According to the doc it is "Return a list of known open source licenses."

How to get an exhaustive lists of the licences and add it to my main query above to make my research more precise?

I can’t seem to find clear answers as the documentation about the GraphQl api for GitHub is scarce and quite vague.

Thanks

Answered by lee-dohm

May 28, 2019

> One more question: given that search first is limited to 100, what if I want the top 1000?

That requires the use of pagination. See this other topic for a good example of how to make it work.

I hope that helps!

View full answer

lee-dohm · 2019-05-24T17:07:40Z

lee-dohm
May 24, 2019

The search connection in GraphQL uses the same syntax as the search box on github.com. This means that if you can get the results you want on github.com, you can get the same results in GraphQL.

The reason why it returns random results is that you didn’t sort by the number of stars. Your query asked for random JavaScript repositories. In order to sort by the number of stars, you would need to execute:

query{
  search(type: REPOSITORY, query: "language:javascript sort:stars-desc", first:10) {
    userCount
    edges {
      node {
        ... on Repository {
          name
          url
          stargazers {
            totalCount
          }
          owner{
            login
          }
        }
      }
    }
  }
}

Please keep in mind that the number of stars on repos can change significantly and regularly for a number of reasons, so a given search is only a moment-in-time result.

With regard to licenses, the query:

query {
  licenses {
    nickname
    name
    spdxId
  }
}

returned the following for me:

{
  "data": {
    "licenses": [
      {
        "nickname": "GNU AGPLv3",
        "name": "GNU Affero General Public License v3.0",
        "spdxId": "AGPL-3.0"
      },
      {
        "nickname": null,
        "name": "Apache License 2.0",
        "spdxId": "Apache-2.0"
      },
      {
        "nickname": null,
        "name": "BSD 2-Clause \"Simplified\" License",
        "spdxId": "BSD-2-Clause"
      },
      {
        "nickname": null,
        "name": "BSD 3-Clause \"New\" or \"Revised\" License",
        "spdxId": "BSD-3-Clause"
      },
      {
        "nickname": null,
        "name": "Eclipse Public License 2.0",
        "spdxId": "EPL-2.0"
      },
      {
        "nickname": "GNU GPLv2",
        "name": "GNU General Public License v2.0",
        "spdxId": "GPL-2.0"
      },
      {
        "nickname": "GNU GPLv3",
        "name": "GNU General Public License v3.0",
        "spdxId": "GPL-3.0"
      },
      {
        "nickname": "GNU LGPLv2.1",
        "name": "GNU Lesser General Public License v2.1",
        "spdxId": "LGPL-2.1"
      },
      {
        "nickname": "GNU LGPLv3",
        "name": "GNU Lesser General Public License v3.0",
        "spdxId": "LGPL-3.0"
      },
      {
        "nickname": null,
        "name": "MIT License",
        "spdxId": "MIT"
      },
      {
        "nickname": null,
        "name": "Mozilla Public License 2.0",
        "spdxId": "MPL-2.0"
      },
      {
        "nickname": null,
        "name": "The Unlicense",
        "spdxId": "Unlicense"
      }
    ]
  }
}

So MIT is in the list. I’m not sure what you mean by “exhaustive list of licenses”, these are the ones that GitHub recognizes currently.

I hope that helps!

0 replies

Nanoboss · 2019-05-25T14:44:07Z

Nanoboss
May 25, 2019
Author

Hi,

Thanks you for the answer unfortunately it doesn’t work as when I run it it doesn’t return the freecodecamp repo which should be number 1. On top of it it return a different result each time.

{'node': {'name': 'freeCodeCamp', 'url': 'https://github.com/freeCodeCamp/freeCodeCamp', 'stargazers': {'totalCount': 302988}, 'owner': {'login': 'freeCodeCamp'}}}
{'node': {'name': 'react', 'url': 'https://github.com/facebook/react', 'stargazers': {'totalCount': 129798}, 'owner': {'login': 'facebook'}}}
{'node': {'name': 'd3', 'url': 'https://github.com/d3/d3', 'stargazers': {'totalCount': 84884}, 'owner': {'login': 'd3'}}}
{'node': {'name': 'create-react-app', 'url': 'https://github.com/facebook/create-react-app', 'stargazers': {'totalCount': 67927}, 'owner': {'login': 'facebook'}}}
{'node': {'name': 'node', 'url': 'https://github.com/nodejs/node', 'stargazers': {'totalCount': 61488}, 'owner': {'login': 'nodejs'}}}
{'node': {'name': 'axios', 'url': 'https://github.com/axios/axios', 'stargazers': {'totalCount': 59767}, 'owner': {'login': 'axios'}}}
{'node': {'name': 'three.js', 'url': 'https://github.com/mrdoob/three.js', 'stargazers': {'totalCount': 51832}, 'owner': {'login': 'mrdoob'}}}
{'node': {'name': 'jquery', 'url': 'https://github.com/jquery/jquery', 'stargazers': {'totalCount': 51626}, 'owner': {'login': 'jquery'}}}
{'node': {'name': 'webpack', 'url': 'https://github.com/webpack/webpack', 'stargazers': {'totalCount': 48971}, 'owner': {'login': 'webpack'}}}
{'node': {'name': 'material-ui', 'url': 'https://github.com/mui-org/material-ui', 'stargazers': {'totalCount': 47154}, 'owner': {'login': 'mui-org'}}}

The right query result should be using stars:>1600 (the number is arbitrary but relevant in my case).

{'node': {'name': 'freeCodeCamp', 'url': 'https://github.com/freeCodeCamp/freeCodeCamp', 'stargazers': {'totalCount': 302987}, 'owner': {'login': 'freeCodeCamp'}}}
{'node': {'name': 'vue', 'url': 'https://github.com/vuejs/vue', 'stargazers': {'totalCount': 139433}, 'owner': {'login': 'vuejs'}}}
{'node': {'name': 'bootstrap', 'url': 'https://github.com/twbs/bootstrap', 'stargazers': {'totalCount': 133526}, 'owner': {'login': 'twbs'}}}
{'node': {'name': 'react', 'url': 'https://github.com/facebook/react', 'stargazers': {'totalCount': 129798}, 'owner': {'login': 'facebook'}}}
{'node': {'name': 'javascript', 'url': 'https://github.com/airbnb/javascript', 'stargazers': {'totalCount': 85584}, 'owner': {'login': 'airbnb'}}}
{'node': {'name': 'd3', 'url': 'https://github.com/d3/d3', 'stargazers': {'totalCount': 84884}, 'owner': {'login': 'd3'}}}
{'node': {'name': 'react-native', 'url': 'https://github.com/facebook/react-native', 'stargazers': {'totalCount': 77492}, 'owner': {'login': 'facebook'}}}
{'node': {'name': 'create-react-app', 'url': 'https://github.com/facebook/create-react-app', 'stargazers': {'totalCount': 67927}, 'owner': {'login': 'facebook'}}}
{'node': {'name': 'node', 'url': 'https://github.com/nodejs/node', 'stargazers': {'totalCount': 61488}, 'owner': {'login': 'nodejs'}}}
{'node': {'name': 'Font-Awesome', 'url': 'https://github.com/FortAwesome/Font-Awesome', 'stargazers': {'totalCount': 59870}, 'owner': {'login': 'FortAwesome'}}}

The support told me that it’s a timeout problem, which is why the reuslts are inconsistent. The only way to prevent that and have consistent results is to narrow down the number of stars.

I got my licences confused sorry, I think that’s it.

One more question: given that search first is limited to 100, what if I want the top 1000?

Thanks you :slight_smile:

0 replies

lee-dohm · 2019-05-28T21:59:07Z

lee-dohm
May 28, 2019

> One more question: given that search first is limited to 100, what if I want the top 1000?

That requires the use of pagination. See this other topic for a good example of how to make it work.

I hope that helps!

0 replies

Nanoboss · 2019-06-06T06:29:07Z

Nanoboss
Jun 6, 2019
Author

It works perfeclty using pagination thanks you. I divided my searach in 20 queries that will return 50 repositories each time and get feed the next cursor at each run.

query($first: Int!, $query: String!, $type: SearchType!, $after: String)
    {search(first:$first ,query:$query, type:$type, after:$after){
                edges {
                  node {
                    ... on Repository {
                      owner{
                        login
                      }
                      name
                      stargazers {
                        totalCount
                      }
                      forkCount
                      defaultBranchRef{
                        target{
                          ... on Commit{
                            history(first:0){
                              totalCount
                              edges{
                                node{
                                  ... on Commit{
                                    committedDate
                                  }
                                }
                              }
                            }
                          }
                        }
                      }
                      pullRequests(states: OPEN){
                        totalCount
                      }
                    }
                  }
                }
                pageInfo{
                    hasPreviousPage
                    startCursor
                    hasNextPage
                    endCursor
                }
              }
            }
    variables = {
              "first": 50,
              "query": "language:javascript stars:&gt;1600",
              "type": "REPOSITORY",
              "after": &lt;CursorValue&gt;
            }

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GitHub Community

top 10 javascript/opensource repositories ranked by star #24415

{{title}}

Replies: 4 comments

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

GitHub Community

top 10 javascript/opensource repositories ranked by star #24415

Nanoboss May 22, 2019

Replies: 4 comments

lee-dohm May 24, 2019

Nanoboss May 25, 2019 Author

lee-dohm May 28, 2019

Nanoboss Jun 6, 2019 Author

Nanoboss
May 22, 2019

lee-dohm
May 24, 2019

Nanoboss
May 25, 2019
Author

lee-dohm
May 28, 2019

Nanoboss
Jun 6, 2019
Author