Github Action Build Caching

Hi, Does anyone know why Actions are built at runtime on every execution without caching? I am observing prolonged build time because of this. Thank you!

14 Likes

GitHub Actions are extremely flexible to allow you to model whatever kind of build semantics you need. On the other hand, this means that there is a large amount of complexity to contend with. Let me start by making sure I understand what problem you’re describing.

It sounds like what you’re saying is that when you create a GitHub Action that uses a Dockerfile that is stored in a GitHub repository, when that Action is executed, the Docker container is generated each time. If I’m understanding you correctly, then it is true that the Dockerfile is evaluated each time the Action is executed but that doesn’t mean that it is built from scratch with no caching. For example, here is a sample log from a GitHub Action in one of the projects I play around with. An excerpt from the log shows that every step was pulled from the cache:

Step 1/11 : FROM elixir:latest
latest: Pulling from library/elixir
a4d8138d0f6b: Already exists
dbdc36973392: Already exists
f59d6d019dd5: Already exists
aaef3e026258: Already exists
6e454d3b6c28: Already exists
56ddef501213: Already exists
b757f6a814a2: Already exists
9f8747f275cf: Already exists
e068efd83bc5: Already exists
Digest: sha256:32b9e7c1e0731b8605476ed32fd8d38e2e84f468364fecd570d735b50f606078
Status: Downloaded newer image for elixir:latest
 ---> 35b5c13db506
Step 2/11 : LABEL "com.github.actions.name"="Generates Elixir documentation"
 ---> Using cache
 ---> d16c33bb6325
Step 3/11 : LABEL "com.github.actions.description"="Generates Elixir project documentation to be published"
 ---> Using cache
 ---> c686ff2812b2
Step 4/11 : LABEL "com.github.actions.icon"="edit"
 ---> Using cache
 ---> 1b9bfa9bca8c
Step 5/11 : LABEL "com.github.actions.color"="green"
 ---> Using cache
 ---> 817da5d0b307
Step 6/11 : LABEL "repository"="https://github.com/lee-dohm/generate-elixir-docs"
 ---> Using cache
 ---> fc437e4219f3
Step 7/11 : LABEL "homepage"="https://github.com/lee-dohm/generate-elixir-docs"
 ---> Using cache
 ---> b311a865891b
Step 8/11 : LABEL "maintainer"="Lee Dohm <lee-dohm@github.com>"
 ---> Using cache
 ---> e3895426546c
Step 9/11 : RUN apt-get update && apt-get install -y --no-install-recommends build-essential
 ---> Using cache
 ---> 3871c7afdfd0
Step 10/11 : COPY ./entrypoint.sh .
 ---> Using cache
 ---> 98368a478307
Step 11/11 : ENTRYPOINT ["/entrypoint.sh"]
 ---> Using cache
 ---> 6ad7298b48c5
Successfully built 6ad7298b48c5

Note the ---\> Using cache lines.

What this does is gives a certain amount of flexibility to the Action. For example, the Action that I created to perform the above task uses a Docker image that includes the runtime for the programming language Elixir. You can see that I defined the Docker image to use the latest version of Elixir. This means that every time the Action executes, it automatically updates my Action to use the latest version of Elixir to run on (since it’s only generating documentation, this isn’t a big risk). If I didn’t want that, I could specify that I want it to generate a Docker container using a more specific version of Elixir, for example v1.8 which is currently equivalent to v1.8.2. Or I could instruct it to use a specific old version of Elixir, such as v1.6.3. This means that it wouldn’t automatically update for new versions of the Elixir runtime, but  would update if I changed certain things in my lee-dohm/generate-elixir-docs repository, such as the entrypoint.sh file.

But let’s say that I don’t want my Action to even evaluate the Dockerfile every time the Action runs, because I want the absolute fastest runtime possible. I want a predefined Docker container to be spun up and get right to work. You can have that too! If you create a Docker image and upload it to Docker Hub or another public registry, you can instruct your Action to use that Docker image specifically using the docker:// form in the uses key. See the GitHub Actions documentation for specifics. This requires a bit more up-front work and maintenance, but if you don’t need the flexibility that the above Dockerfile evaluation system provides, then it may be worth the tradeoff.

I hope that helps! Let us know if you have more questions.

8 Likes

Hi @lee-dohm ,

Thanks for the thorough reply. I can confirm that the previous HCL syntax version of Actions does show “Using cache” when building from a Dockerfile as seen here. However, since upgrading to the yml workflows, it stopped caching, as seen in the lack of “Using cache” here

21 Likes

does a solution for the problem exist? as the answer does not really seem to fix the issue?

4 Likes

@lee-dohmthe docker image steps in my actions don’t seem to be cached. @wei said that this started happening with the transition to yml workflows.

4 Likes

Same problem here, it does not cache the layers at all, every push is a new build from start 

3 Likes

Also ran into this, my builds reguarly take 6 minutes now because the container image is built from scratch (caching used to work when Actions was in early preview)

6 Likes

This is really NOT a solution. The solution is to support Docker build cache (layer caching).

9 Likes

There is a new way of doing this. https://github.com/actions/cache

But i cant find a working example of using this action with docker-compose.

Yeah, I was also checking the same repo. They have built caching for generic store. Not for Docker Layer Caching.

2 Likes

How to use cahing on Ruby Gems?

It always install all gems …

+1

I know “+1”'s are annoying, but so are “solutions” for questions that are “solved” when they really aren’t. :stuck_out_tongue:

5 Likes

For solve to this probelm, this action is proper to solve this problem. It cache the layers on registry.

The way actions building works during a workflow run for actions taken from the Github Actions Marketplace seems completely bizarre to me. I mean, when you add a step that uses a marketplace action (at a specific version), the workflow automatically adds an initial step on execution to build that action, from scratch, re-running the Docker compile step every time I build. The vast majority of time spent in my workflows is for building actions from the Marketplace. To me this is completely unexpected behavior and wastes a ton of time. The whole point of the Marketplace is to allow people to reuse actions, so why do we have to build those actions from scratch every single time, with no caching?

6 Likes

While trying to workaround the PR Secrets (https://github.community/t5/GitHub-Actions/Make-secrets-available-to-builds-of-forks/td-p/30678) problem, had the bright idea of cacheing the credentials in a container.  Just a guess, but cacheing can never be enabled because of the security implications discussed in the linked community.  Really wish someone would just call this out and close these types of things as ‘By Design’.

if you’re not too hung up on GitHub hosted runners, you can think about a self hosted runner. Then you have control over your runner’s cache.

https://docs.github.com/en/actions/hosting-your-own-runners/about-self-hosted-runners

Could you please point out how to use the caching with the self hosted runner? Specifically with docker.