Audit Event Logs getting delayed with REST API

I am using REST-API to query the Audit Logs periodically. I noticed sometimes the workflow-related audit logs get delayed. The log entry does not show up on the first run but appears with the second run even though the event timestamp is lower than the most recent one. I query the logs in desc order and notice the behavior with recently generated events.

Is there documentation on how long an event can be delayed? Does it provide an SLA of how much time it can take from the moment an event happens until it appears through the REST-API?

UPDATE:
The example query for the REST-API looks like:
https://api.github.com/organizations/xxxx/audit-log?order=desc&include=allper_page=100&after=&&before=

The response I see in two invocations are follows:

First invocation: 
{
  ...
  { "@timestamp": 100 ... },
  { "@timestamp": 99  ... },
  { "@timestamp": 97  ... },
  { "@timestamp": 96  ... },
  ...
}
Second invocation: 
{
  ...
  { "@timestamp": 100... },
  { "@timestamp": 99  ... },
  { "@timestamp": 98  ... },
  { "@timestamp": 97  ... },
  { "@timestamp": 96  ... },
  ...
}

The events with @timestamp: 98 is missing from the first response and it appears in the next response. The timestamp here are dummy and created on based on my observations.

2 Likes

Hi @kumarak,

I believe this will be due to the nature of processing, the behaviour is not limited to events related to workflows, but other events also, I observed this in bot the beta and GA versions.
You can see this by performing an auditable event change such as permission change and immediately trying to view it in the audit event, you may or may not see it and on occasion it may take a minute before viewable, I do not know the exact time, doubt GitHub will commit to an SLA as such, but know harm in asking the question as you have done above…

Hi @byrneh,

Thanks for the response. I agree the behaviour is not limited to workflow events. The other event logs can also get delayed. I noticed it with my test setup which is limited. I run the script looking for the new logs every minute. I am curious if there is a safe delay time that I can assume and all events will appear through REST-API. How worse can it get if the script is deployed with a real workload generating thousands of events every minute?