Run actions in a container with guaranteed CPU

Code to Cloud GitHub Actions
#1

In the upstream Prometheus and Thanos projects we want to run some short function benchmarks in a Github actions and wondering if you could an an option to run an action in a container with a guratanteed CPU.

We already run some tests and at the moment running the same benchmark in the same step has quite inconsistent results. For example sometimes it will take 5 sec to complete a bench tests and others it will take 6sec which is a big difference and can’t rely on the results.

2 Likes
#2

Hi @krasi-georgiev ,

The container will run any steps in a job that don’t already specify a container, and container options

 support cpu options like below, please refer to offical doc for more details.

jobs:
  my_job:
    container:
      image: node:10.16-jessie
      env:
        NODE_ENV: development
      ports:
        - 80
      volumes:
        - my_docker_volume:/volume_mount
      options: --cpus 1

Thanks.

#3

What if we don’t run inside a container ? Can I specify this option ?

like

- name: launch tests
  run: go test .
  options: --cpus 4
#4

Just to clarify that using the suggested option didn’t change anything and the CPU time is not dedicated.

Look at the bench  results here

https://github.com/nevill/prometheus/runs/502177828?check_suite_focus=true

open the print logs step and look at the test called BenchmarkRangeQuery/expr=holt\_winters(a\_hundred[1d],\_0.3,\_0.3),steps=1000-2 sometime it takes 5985624324ns and sometimes 6334650167ns which is like 4sec different.

When we run the same test on a dedicated machine there is no difference at all and on GKE only 1% diff which is ok.

It seems that you don’t provide any CPU isolation neither for VMs or docker containers. So please take this issue as a feature request to add this as an option.

#5

Hi @nevill , no, you cannot. You can set it as env and invoke it in command as an alternative.

Hi @krasi-georgiev , according to policy, please raise your request here where github product manager will take a review.

Thanks.

#6

Sure will do, but am I correct in the assumption that there is no way to requests dedicated CPU time with the current GH actions?

#7

Hi @krasi-georgiev ,

All the docker create related options are listed here, looks there’s no option to request a ‘delicated CPU time’.

Since it’s fine on your dedicated machine, it’s recommeded to set it as local self-hosted runner instead. In the meanwhile, you can raise a feature_request ticket.

Thanks.

#8

I have requested it as a feature. Thanks

1 Like
#9

Another approach is to use Cachegrind, which simulates a CPU and gives you an instruction count for your program, so it can give consistent results across multiple CI runs. SQLite uses this technique for performance benchmarking (https://sqlite.org/cpu.html#performance_measurement).

I wrote an article explaining this in far more detail, if you’re interested: https://pythonspeed.com/articles/consistent-benchmarking-in-ci/

1 Like