Secondary Rate Limit - Code Search REST api

Hello Everyone,

I am trying to keep my requests under 30req/min and still getting blocked by secondary rate limit. There is also no Retry-After Header as I am just searching and not creating content.

I am doing authenticated requests with token and valid User-agent fields. I try to modify my query so that the response time is with 500ms by reducing the per_page parameter value.

Here is the 403 Response.

{
“documentation_url”: “Resources in the REST API - GitHub Docs”,
“message”: “You have exceeded a secondary rate limit. Please wait a few minutes before you try again.”
}

Any nudge how to handle such scenarios or suggestions ? Thank you for your time.

Hey @Auror007 !

There is a section in the Docs specifically dedicated to this:

https://docs.github.com/en/rest/guides/best-practices-for-integrators#dealing-with-secondary-rate-limits

1 Like

Hey @mpboom , Thanks for your help. I am already following these best practices and still there is no indication in response (Like Retry-After ) which tells me to reduce my time between two requests ( currently sending at 20req/min authenticated requests). The 403 error above appears just randomly.

I have even tried reading the response time and reducing the per_page parameter because I thought it might be overloading the cpu in the server. But still no success.

1 Like

You might be confusing the “normal” rate limit with secondary rate limits.

Indeed, the normal rate limit is documented, strictly enforced and returns helpful headers like Retry-After. However, the secondary rate limit is something different:
https://docs.github.com/en/rest/overview/resources-in-the-rest-api#secondary-rate-limits

Quoting from the docs:

In order to provide quality service on GitHub, additional rate limits may apply to some actions when using the API. For example, using the API to rapidly create content, poll aggressively instead of using webhooks, make multiple concurrent requests, or repeatedly request data that is computationally expensive may result in secondary rate limiting.

Hence, it would be hard for me and you to tell what exactly is causing it (e.g. saying “I only perform X requests per minute” isn’t sufficient in this case, as per the snippet above). Being a good API citizen reduces the chances of hitting a secondary rate limit.

What API endpoint are you requesting data from?

1 Like

Oh! Thanks for the clarification. The endpoint is https://api.github.com/search/code

For instance, try querying this with an auth token :
https://api.github.com/search/code?q=CVE-2022-25636+in:file&type=Code&per_page=50&sort=indexed&page=1

Ah, I see you’re fetching the code search endpoint. While I can’t be sure, I think this falls under:

repeatedly request data that is computationally expensive

Therefore you might hit the secondary rate limiting.

I don’t really have any solid advice, though. Since code search is very expensive in terms of computational power I’m not surprised. The only advice I can give is to try and search a little less or try capturing all results in one, more general, search. I don’t think that for these actions rate limits of “x” requests/hr are documented.

1 Like

This post was flagged by the community and is temporarily hidden.