Unclear what limit was reached at LFS

GitHub Ecosystem REST API
#1

Original issue is github-lfs rate limit exceeded · Issue #6761 · datalad/datalad · GitHub which have more details. But overall

response details with X-RateLimit indicators which seems to not explain "why" 
HttpExceptionRequest Request {
    host                 = "github.com"
    port                 = 443
    secure               = True
    requestHeaders       = [("Authorization","<REDACTED>"),("Accept","application/vnd.git-lfs+json"),("Content-Type","application/vnd.git-lfs+json"),("User-Agent","git-annex/10.20220504")]
    path                 = "/dnkennedy/test_share.git/info/lfs/objects/batch"
    queryString          = ""
    method               = "POST"
    proxy                = Nothing
    rawBody              = False
    redirectCount        = 10
    responseTimeout      = ResponseTimeoutDefault
    requestVersion       = HTTP/1.1
    proxySecureMode      = ProxySecureWithConnect
  }
   (StatusCodeException (Response {responseStatus = Status {statusCode = 403, statusMessage = "Forbidden"}, responseVersion = HTTP/1.1, responseHeaders = [("Access-Control-Allow-Origin","*"),("Access-Control-Expose-Headers","ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset"),("Content-Encoding","gzip"),("Content-Security-Policy","default-src 'none'"),("Content-Type","application/json; charset=utf-8"),("Date","Fri, 10 Jun 2022 16:49:48 GMT"),("github-authentication-token-expiration","2022-08-18 02:22:40 UTC"),("Referrer-Policy","origin-when-cross-origin, strict-origin-when-cross-origin"),("request-hmac","XXXX"),("Server","GitHub.com"),("Strict-Transport-Security","max-age=31536000; includeSubdomains; preload"),("Vary","Accept-Encoding, Accept, X-Requested-With"),("X-Accepted-OAuth-Scopes","repo"),("X-Content-Type-Options","nosniff"),("X-Frame-Options","deny"),("X-GitHub-Media-Type","unknown"),("X-OAuth-Scopes","repo"),("X-RateLimit-Limit","3000"),("X-RateLimit-Remaining","2998"),("X-RateLimit-Reset","1654879846"),("X-RateLimit-Resource","lfs"),("X-RateLimit-Used","2"),("X-XSS-Protection","0"),("Content-Length","103"),("X-GitHub-Request-Id","XXXX")], responseBody = (), responseCookieJar = CJ {expose = []}, responseClose' = ResponseClose, responseOriginalRequest = Request {
    host                 = "github.com"
    port                 = 443
    secure               = True
    requestHeaders       = [("Authorization","<REDACTED>"),("Accept","application/vnd.git-lfs+json"),("Content-Type","application/vnd.git-lfs+json"),("User-Agent","git-annex/10.20220504")]
    path                 = "/dnkennedy/test_share.git/info/lfs/objects/batch"
    queryString          = ""
    method               = "POST"
    proxy                = Nothing
    rawBody              = False
    redirectCount        = 10
    responseTimeout      = ResponseTimeoutDefault
    requestVersion       = HTTP/1.1
    proxySecureMode      = ProxySecureWithConnect
  }
  }) "{\"message\":\"Rate Limit Exceeded\",\"documentation_url\":\"https://support.github.com/contact\"}")
rate limit query result for the user which also says that there is plenty left 
{
  "resources": {
    "core": {
      "limit": 5000,
      "used": 2,
      "remaining": 4998,
      "reset": 1654881374
    },
    "search": {
      "limit": 30,
      "used": 0,
      "remaining": 30,
      "reset": 1654880689
    },
    "graphql": {
      "limit": 5000,
      "used": 0,
      "remaining": 5000,
      "reset": 1654884229
    },
    "integration_manifest": {
      "limit": 5000,
      "used": 0,
      "remaining": 5000,
      "reset": 1654884229
    },
    "source_import": {
      "limit": 100,
      "used": 0,
      "remaining": 100,
      "reset": 1654880689
    },
    "code_scanning_upload": {
      "limit": 1000,
      "used": 0,
      "remaining": 1000,
      "reset": 1654884229
    },
    "actions_runner_registration": {
      "limit": 10000,
      "used": 0,
      "remaining": 10000,
      "reset": 1654884229
    },
    "scim": {
      "limit": 15000,
      "used": 0,
      "remaining": 15000,
      "reset": 1654884229
    }
  },
  "rate": {
    "limit": 5000,
    "used": 2,
    "remaining": 4998,
    "reset": 1654881374
  }

and in billing it also doesn’t say that anything exceeded:

image
image1574×550 28.3 KB

So – why he was rate limited???

Thanks in advance for the ideas!

1 Like
#2

The rate limit probably has nothing to do with bandwidth/storage quotas. The error itself suggests a few possible causes and fixes:

This could have failed because --fast is enabled.

"message\":\"Rate Limit Exceeded\"

Just wait for a few hours and see if the issue persists. Otherwise, get in touch as the error message suggests:

\"documentation_url\":\"https://support.github.com/contact\"

Usually, you shouldn’t notice rate limits like these. It might be worth it to check your toolchain and workflows to see if you’re somewhere making a large amount of requests.

1 Like
#3

that one is coming from git-annex I believe as a generic “hint” on why annex operation would have failed but it would not be related to the rate limit (it is not like annex goes “faster” it just disables some checks etc).

well - we waited for a day – the same. And it is the question – which limit is exceeded, since as I showed above – seems no limits were exceeded.

I will try that contact now.

1 Like