Speed up installation of prerequisites?

Many actions require steps like installing apt packages or downloading and installing CLIs or build related tools that are prerequisites to the primary tasks to be done within a GitHub action.

Typically these prerequisites don’t change very often yet the time must be spent installing them over and over each time the repo contents frequently change. Are there techniques to help mitigate this?

For example, jobs execute in parallel so if there was a way to tie Job 1 (“prereqs”) as a dependency of Step 6 of job 2 (but not job 2 as a whole), then steps 1-5 of job 2 could start executing along with job 1 and job 1 would likely complete before step 6 of job 2 was reached. If not, step 6 of job 2 would wait until job 1 completes. AFAIK this isn’t possible today? - I’d have to execute things sequentially in the same job or have job 2 depend on job 1 (same result?) or take a chance that job 1 may not be done before some part of it is needed in job 2.

Or if there was some way to have a pool of agents with steps from a job that can be cached in some way so they are not executed each time?

I’m pretty sure you can cache them. I’ll see if I can dig up some info.

1 Like

You want to use the cache action. Here’s an example using it to cache ruby gems (but it’s generic and can cache anything.

The key thing is to use a hashkey that makes sense so the cache is invalidated when the things you are caching change. In this case below, I’m using a hash of the Gemfile.lock (which is the lock file for Ruby gems). When it changes, the cache is invalidated.

name: Tests

on: pull_request

jobs:
  rake:
    name: rake tests
    runs-on: ubuntu-latest

    # runs all of the steps inside the specified container rather than on the VM host.  
    # Because of this the network configuration changes from host based network to a container network.
    container:
      image:  ruby:2.6.6

    steps:
    - uses: actions/checkout@v2
      with:
        path: app/

    # Need 2 caches for the gems.  The cache key is based on a hash of the gemspec file
    # When the gemspec file changes, the cache is invalidated
    - uses: actions/cache@v2
      with:
        path: app/vendor/cache
        key: ${{ runner.os }}-gem-${{ hashFiles('**/Gemfile.lock') }}
        restore-keys: |
          ${{ runner.os }}-gem-

    - uses: actions/cache@v2
      with:
        path: /usr/local/bundle
        key: ${{ runner.os }}-bundle-${{ hashFiles('**/Gemfile.lock') }}
        restore-keys: |
          ${{ runner.os }}-bundle-

    - name: bundle
      working-directory: ./app
      run: |
        bundle install --jobs 4 --retry 3

@dnorth98 Thanks, will take a look at that. I’m not sure that would help for apt packages that may place files in varying locations with various dependencies but for tools or app package dependencies that I control the destination more that may be of use.

There’s:

1 Like

A Docker image would be another possibility as long as Linux only is fine for your project. You can containerize an entire build environment including dependencies and it’s just a single, rather fast pull. It probably doesn’t beat a cache though.