How to deal with abuse rate limiting when adding a large amount of comments

Hi,

I’m currently writing an integration of our CI/CD server product (Ikan ALM, https://www.ikanalm.com/) with GitHub as an Issue tracking system. we provide automatic parsing of commit comments for issue IDs, retrieving the details for these issues from the relevant API, as well as the ability to automatically add comments to an issue on a successful server side build or deploy with some details about the build or deploy.

Now, because comments on GitHub issues cause notifications to go out, these have abuse rate limit rules. Generally speaking a build or deploy will only have a handful of issues, maybe a dozen or so. However, in some extreme cases, like a deploy stage that hasn’t been activated in years, there may be 500 or more issues associated with a single build or deploy, and triggering comment notifications on 500 issues is probably a really good way to trigger abuse rate limits.

I’m already requiring the use of a Personal Access Token using the header, but I don’t know how much that will help. Would it be enough to add a fixed delay of 2-3 seconds between creating comments? Or would there be a more appropriate way to deal with this?

:wave: hello there @ikannak, and welcome to the GitHub Support Community!

Thanks for reaching out and sorry to hear you’ve been having trouble. Based on your description, it seems that you’re running into the abuse rate limits.

Creating issues and comments via the API triggers email notifications just like they’d be triggered when you create content via the Web UI. The abuse rate limit was introduced in order to prevent spikes in notification emails which cause problems for users and our infrastructure.

Instead of using the regular issues API, there’s another API endpoint that our team worked on which was designed with such migrations in mind:

  • It should allow you to preserve the dates of issues and comments that you migrate.
  • It doesn’t trigger notifications for each issue and comment that’s created.
  • It isn’t affected by the content-creation abuse rate limits, which would normally cause trouble if you attempt to create lots of content in a short period of time.

Note that this API doesn’t currently allow you to create issues authored by other users, since that would be a security concern. It’s something we’re considering adding in the future by allowing users to opt-in to such imports, but I can’t say when that will happen. For now, you could add that information in a short header/footer in the content you create (e.g., at the bottom of each issue and comment).

Our team wrote up a walkthrough of using this API in this gist:

I’m wondering if you’d be up to give that a try with a few issues and a test project to see if it fits your needs?

A quick update: I’ve just done some testing, and I can’t seem to trigger the rate limiting on adding comments. As in, I’ve added a thousand comments to the same issue in about 830 seconds without even waiting between attempts, just as fast as they’re going, and even adding the issues from a different account than the account used to create the issue, but even adding 1000 comments like that doesn’t seem to trigger any rate limiting, even though I still get emails for each issue.

I had the same issue when I was trying to test rate limiting on Jira in the cloud: no matter what I tried, no rate limiting was triggered.