Hi, Does anyone know why Actions are built at runtime on every execution without caching? I am observing prolonged build time because of this. Thank you!
GitHub Actions are extremely flexible to allow you to model whatever kind of build semantics you need. On the other hand, this means that there is a large amount of complexity to contend with. Let me start by making sure I understand what problem you’re describing.
It sounds like what you’re saying is that when you create a GitHub Action that uses a
Dockerfile that is stored in a GitHub repository, when that Action is executed, the Docker container is generated each time. If I’m understanding you correctly, then it is true that the
Dockerfile is evaluated each time the Action is executed but that doesn’t mean that it is built from scratch with no caching. For example, here is a sample log from a GitHub Action in one of the projects I play around with. An excerpt from the log shows that every step was pulled from the cache:
Step 1/11 : FROM elixir:latest latest: Pulling from library/elixir a4d8138d0f6b: Already exists dbdc36973392: Already exists f59d6d019dd5: Already exists aaef3e026258: Already exists 6e454d3b6c28: Already exists 56ddef501213: Already exists b757f6a814a2: Already exists 9f8747f275cf: Already exists e068efd83bc5: Already exists Digest: sha256:32b9e7c1e0731b8605476ed32fd8d38e2e84f468364fecd570d735b50f606078 Status: Downloaded newer image for elixir:latest ---> 35b5c13db506 Step 2/11 : LABEL "com.github.actions.name"="Generates Elixir documentation" ---> Using cache ---> d16c33bb6325 Step 3/11 : LABEL "com.github.actions.description"="Generates Elixir project documentation to be published" ---> Using cache ---> c686ff2812b2 Step 4/11 : LABEL "com.github.actions.icon"="edit" ---> Using cache ---> 1b9bfa9bca8c Step 5/11 : LABEL "com.github.actions.color"="green" ---> Using cache ---> 817da5d0b307 Step 6/11 : LABEL "repository"="https://github.com/lee-dohm/generate-elixir-docs" ---> Using cache ---> fc437e4219f3 Step 7/11 : LABEL "homepage"="https://github.com/lee-dohm/generate-elixir-docs" ---> Using cache ---> b311a865891b Step 8/11 : LABEL "maintainer"="Lee Dohm <email@example.com>" ---> Using cache ---> e3895426546c Step 9/11 : RUN apt-get update && apt-get install -y --no-install-recommends build-essential ---> Using cache ---> 3871c7afdfd0 Step 10/11 : COPY ./entrypoint.sh . ---> Using cache ---> 98368a478307 Step 11/11 : ENTRYPOINT ["/entrypoint.sh"] ---> Using cache ---> 6ad7298b48c5 Successfully built 6ad7298b48c5
---\> Using cache lines.
What this does is gives a certain amount of flexibility to the Action. For example, the Action that I created to perform the above task uses a Docker image that includes the runtime for the programming language Elixir. You can see that I defined the Docker image to use the latest version of Elixir. This means that every time the Action executes, it automatically updates my Action to use the latest version of Elixir to run on (since it’s only generating documentation, this isn’t a big risk). If I didn’t want that, I could specify that I want it to generate a Docker container using a more specific version of Elixir, for example v1.8 which is currently equivalent to v1.8.2. Or I could instruct it to use a specific old version of Elixir, such as v1.6.3. This means that it wouldn’t automatically update for new versions of the Elixir runtime, but would update if I changed certain things in my lee-dohm/generate-elixir-docs repository, such as the
But let’s say that I don’t want my Action to even evaluate the Dockerfile every time the Action runs, because I want the absolute fastest runtime possible. I want a predefined Docker container to be spun up and get right to work. You can have that too! If you create a Docker image and upload it to Docker Hub or another public registry, you can instruct your Action to use that Docker image specifically using the
docker:// form in the
uses key. See the GitHub Actions documentation for specifics. This requires a bit more up-front work and maintenance, but if you don’t need the flexibility that the above
Dockerfile evaluation system provides, then it may be worth the tradeoff.
I hope that helps! Let us know if you have more questions.
Hi @lee-dohm ,
Thanks for the thorough reply. I can confirm that the previous HCL syntax version of Actions does show “Using cache” when building from a Dockerfile as seen here. However, since upgrading to the yml workflows, it stopped caching, as seen in the lack of “Using cache” here.
does a solution for the problem exist? as the answer does not really seem to fix the issue?
@lee-dohmthe docker image steps in my actions don’t seem to be cached. @wei said that this started happening with the transition to yml workflows.
Same problem here, it does not cache the layers at all, every push is a new build from start
Also ran into this, my builds reguarly take 6 minutes now because the container image is built from scratch (caching used to work when Actions was in early preview)
This is really NOT a solution. The solution is to support Docker build cache (layer caching).
There is a new way of doing this. https://github.com/actions/cache
But i cant find a working example of using this action with docker-compose.
Yeah, I was also checking the same repo. They have built caching for generic store. Not for Docker Layer Caching.
How to use cahing on Ruby Gems?
It always install all gems …
I know “+1”'s are annoying, but so are “solutions” for questions that are “solved” when they really aren’t.
For solve to this probelm, this action is proper to solve this problem. It cache the layers on registry.
The way actions building works during a workflow run for actions taken from the Github Actions Marketplace seems completely bizarre to me. I mean, when you add a step that uses a marketplace action (at a specific version), the workflow automatically adds an initial step on execution to build that action, from scratch, re-running the Docker compile step every time I build. The vast majority of time spent in my workflows is for building actions from the Marketplace. To me this is completely unexpected behavior and wastes a ton of time. The whole point of the Marketplace is to allow people to reuse actions, so why do we have to build those actions from scratch every single time, with no caching?
While trying to workaround the PR Secrets (https://github.community/t5/GitHub-Actions/Make-secrets-available-to-builds-of-forks/td-p/30678) problem, had the bright idea of cacheing the credentials in a container. Just a guess, but cacheing can never be enabled because of the security implications discussed in the linked community. Really wish someone would just call this out and close these types of things as ‘By Design’.
if you’re not too hung up on GitHub hosted runners, you can think about a self hosted runner. Then you have control over your runner’s cache.
Could you please point out how to use the caching with the self hosted runner? Specifically with docker.