Custom Project Insights for CI/CD

As part of our CI/CD pipeline using GitHub Actions we get a bunch of numbers, some from build, some from tests, etc. that we would like to keep track of (ideally visually) in between builds, for example to ensure they don’t increase suddenly without a reason. Examples of such values are:

  • build time
  • binary size
  • run time of a test
  • values printed/returned by tests
  • etc.

Does GitHub Actions have any way to record and visualize such custom values on every commit? If no, does anyone know of any readily available tools we can use?

We were looking whether Firebase has some kind of command-line solution that would allow us report events into their analytics, but it doesn’t seem to have such. If there is an easy way to simply dump those values into a cloud spreadsheet or database, that would be good enough as well right now. Any suggestions will be greatly appreciated! Thank you!

P.S. The repository is private, so ideally we wouldn’t want any source exposure to 3rd parties

P.P.S. For those of you with a similar request, I’ve also posted this question on Stack Overflow, so you might want to check CI/CD Analytics for custom values in GitHub Actions thread there.

Thanks for your feedback. As far as I know, there are no existing actions in Github Action to meet your needs. You can try to add custom action to develop these functions at the end of all jobs.
In this custom action: if you want to get the running time of each job, you can get the run_id through Github Context in the current run, and then use " List jobs for a workflow run" REST API to get the start time and end time of each job. Regarding the binary size and test results, you could try to use the command line to obtain and output them to the specified file. Hope it helps you.

1 Like

Hi @niconbw!

Thank you for your reply! My problem is not in getting the timings or the binary sizes - i can easily get those with available OS and scripting tools. My problem is getting those numbers outside of a given workflow run so that I can compare those numbers historically over many workflow runs. For example I want to be able to see if there was any significant binary size change over the past month. I have the numbers on each workflow run already, I am just looking where and how to persist and ideally visualize them. GitHub’s Project Insights seems to be an ideal place for keeping such custom metrics, but I don’t see any way of recording custom insights in there. I’m not hung up on project insights though, so if someone can point me to a quick and easy way of getting these numbers elsewhere, I would be more than happy.

Thank you!
Yuriy

Hi @solodon4

It seems that you can not get the above information from historical data over the past month. But you can try to upload Artifacts during each workflow run from now on, so you can compare file sizes through " List artifacts for a repository" REST API to check if there was any significant binary size change in the next month. By the way, you can save response of the above REST API to the specified file.
PS. Artifacts automatically expire after 90 days, but you can always reclaim used GitHub Actions storage by deleting artifacts before they expire on GitHub.

Hi @niconbw!

Thanks again for the reply, but again, that is not the problem I’m having. Ignore binary sizes - it was just one of the examples of the numbers. My workflow run produces some numbers every time I run it. I would like to persist and visualize those numbers in between workflow runs. What those numbers are is immaterial, i just want a historic log per each of the numbers that I can see (ideally visually) over extended period of time of running my workflows.

Ideally, i would like to be able to see those numbers in projects insights as custom values. Project insights shouldn’t care how i got those numbers, just let me log them on each workflow run under a given name and then visualize historic values per each unique name.

Hi @solodon4
Currently, there is no build-in feature to record and visualize such custom values on every commit in GitHub Action. You could directly report a feature request here. That will allow you to directly interact with the appropriate engineering team, and make it more convenient for the engineering team to collect and categorize your suggestions.