Help
cancel
Showing results for 
Search instead for 
Did you mean: 
Copilot Lvl 3
Message 1 of 5

Insight into VM performance

Two of our applications now have CI jobs running on Actions. The setup was easy and we're looking forward to using Actions for more applications.

 

We noticed that the test suites currently take a lot longer to run when compared to Circle CI. To a large degree this is attributable to a lack of dependency caching (which we're glad to see is already being addressed). However, we have also noticed that the test suite itself takes notably longer (3m05s on Circle vs 5m15s on Actions)

 

Is this simply a lower spec machine that Actions are running on? Circle runs on a 2CPU/4096MB box.

 

It probably doesn't make a difference, but we're running Rails rspec tests. I know its still early days for Github actions, so perhaps comparisons like this are just premature.

4 Replies
GitHub Staff
Message 2 of 5

Re: Insight into VM performance

The virtual machines are 2 cores with 7 GB RAM - for Windows and Linux these are the "Standard_DS2_v2" VM types in Azure. https://help.github.com/en/articles/virtual-environments-for-github-actions

 

I don't know offhand why you would see different performance characteristics (beyond the caching, of course) but it would be interesting to understand your workload to understand why it's slower on Actions.

Copilot Lvl 3
Message 3 of 5

Re: Insight into VM performance

That is interesting.

 

The only thing I can think of is that our tests are currently quite chatty with the postgres database (which we are running as a service). Perhaps there is some kind of network latency to the db service. When I get a chance I will try do another benchmark with only tests that dont hit the database and see if that makes a difference.

GitHub Staff
Message 4 of 5

Re: Insight into VM performance

Aha, interesting.  Also, just curious about an apples-to-apples sort of comparison: how are you running Postgres on your other CI provider?  If it's not in a container, I wonder if there's a possible latency there, as well.

Copilot Lvl 3
Message 5 of 5

Re: Insight into VM performance

We're running Postgres in the same way on our other provider. Apart from the small differences in their schemas, the CI yml files look effectively identical. If you'd like I can share those yml files here