Github Action Build Caching #25971

wei · 2019-07-17T19:45:29Z

wei
Jul 17, 2019

Hi, Does anyone know why Actions are built at runtime on every execution without caching? I am observing prolonged build time because of this. Thank you!

Answered by lee-dohm

Jul 22, 2019

GitHub Actions are extremely flexible to allow you to model whatever kind of build semantics you need. On the other hand, this means that there is a large amount of complexity to contend with. Let me start by making sure I understand what problem you’re describing.

It sounds like what you’re saying is that when you create a GitHub Action that uses a Dockerfile that is stored in a GitHub repository, when that Action is executed, the Docker container is generated each time. If I’m understanding you correctly, then it is true that the Dockerfile is evaluated each time the Action is executed but that doesn’t mean that it is built from scratch with no caching. For example, here is a sample log …

View full answer

lee-dohm · 2019-07-22T21:16:37Z

lee-dohm
Jul 22, 2019

GitHub Actions are extremely flexible to allow you to model whatever kind of build semantics you need. On the other hand, this means that there is a large amount of complexity to contend with. Let me start by making sure I understand what problem you’re describing.

It sounds like what you’re saying is that when you create a GitHub Action that uses a Dockerfile that is stored in a GitHub repository, when that Action is executed, the Docker container is generated each time. If I’m understanding you correctly, then it is true that the Dockerfile is evaluated each time the Action is executed but that doesn’t mean that it is built from scratch with no caching. For example, here is a sample log from a GitHub Action in one of the projects I play around with. An excerpt from the log shows that every step was pulled from the cache:

Step 1/11 : FROM elixir:latest
latest: Pulling from library/elixir
a4d8138d0f6b: Already exists
dbdc36973392: Already exists
f59d6d019dd5: Already exists
aaef3e026258: Already exists
6e454d3b6c28: Already exists
56ddef501213: Already exists
b757f6a814a2: Already exists
9f8747f275cf: Already exists
e068efd83bc5: Already exists
Digest: sha256:32b9e7c1e0731b8605476ed32fd8d38e2e84f468364fecd570d735b50f606078
Status: Downloaded newer image for elixir:latest
 ---> 35b5c13db506
Step 2/11 : LABEL "com.github.actions.name"="Generates Elixir documentation"
 ---> Using cache
 ---> d16c33bb6325
Step 3/11 : LABEL "com.github.actions.description"="Generates Elixir project documentation to be published"
 ---> Using cache
 ---> c686ff2812b2
Step 4/11 : LABEL "com.github.actions.icon"="edit"
 ---> Using cache
 ---> 1b9bfa9bca8c
Step 5/11 : LABEL "com.github.actions.color"="green"
 ---> Using cache
 ---> 817da5d0b307
Step 6/11 : LABEL "repository"="https://github.com/lee-dohm/generate-elixir-docs"
 ---> Using cache
 ---> fc437e4219f3
Step 7/11 : LABEL "homepage"="https://github.com/lee-dohm/generate-elixir-docs"
 ---> Using cache
 ---> b311a865891b
Step 8/11 : LABEL "maintainer"="Lee Dohm <lee-dohm@github.com>"
 ---> Using cache
 ---> e3895426546c
Step 9/11 : RUN apt-get update && apt-get install -y --no-install-recommends build-essential
 ---> Using cache
 ---> 3871c7afdfd0
Step 10/11 : COPY ./entrypoint.sh .
 ---> Using cache
 ---> 98368a478307
Step 11/11 : ENTRYPOINT ["/entrypoint.sh"]
 ---> Using cache
 ---> 6ad7298b48c5
Successfully built 6ad7298b48c5

Note the ---\> Using cache lines.

What this does is gives a certain amount of flexibility to the Action. For example, the Action that I created to perform the above task uses a Docker image that includes the runtime for the programming language Elixir. You can see that I defined the Docker image to use the latest version of Elixir. This means that every time the Action executes, it automatically updates my Action to use the latest version of Elixir to run on (since it’s only generating documentation, this isn’t a big risk). If I didn’t want that, I could specify that I want it to generate a Docker container using a more specific version of Elixir, for example v1.8 which is currently equivalent to v1.8.2. Or I could instruct it to use a specific old version of Elixir, such as v1.6.3. This means that it wouldn’t automatically update for new versions of the Elixir runtime, but would update if I changed certain things in my lee-dohm/generate-elixir-docs repository, such as the entrypoint.sh file.

But let’s say that I don’t want my Action to even evaluate the Dockerfile every time the Action runs, because I want the absolute fastest runtime possible. I want a predefined Docker container to be spun up and get right to work. You can have that too! If you create a Docker image and upload it to Docker Hub or another public registry, you can instruct your Action to use that Docker image specifically using the docker:// form in the uses key. See the GitHub Actions documentation for specifics. This requires a bit more up-front work and maintenance, but if you don’t need the flexibility that the above Dockerfile evaluation system provides, then it may be worth the tradeoff.

I hope that helps! Let us know if you have more questions.

0 replies

wei · 2019-08-13T18:53:39Z

wei
Aug 13, 2019
Author

Hi @lee-dohm ,

Thanks for the thorough reply. I can confirm that the previous HCL syntax version of Actions does show “Using cache” when building from a Dockerfile as seen here. However, since upgrading to the yml workflows, it stopped caching, as seen in the lack of “Using cache” here.

0 replies

tobsch · 2019-09-10T18:14:44Z

tobsch
Sep 10, 2019

does a solution for the problem exist? as the answer does not really seem to fix the issue?

0 replies

flying-sheep · 2019-09-17T15:29:50Z

flying-sheep
Sep 17, 2019

@lee-dohmthe docker image steps in my actions don’t seem to be cached. @wei said that this started happening with the transition to yml workflows.

0 replies

lehno · 2019-10-04T11:01:25Z

lehno
Oct 4, 2019

Same problem here, it does not cache the layers at all, every push is a new build from start

0 replies

dustinsoftware · 2019-10-19T05:25:33Z

dustinsoftware
Oct 19, 2019

Also ran into this, my builds reguarly take 6 minutes now because the container image is built from scratch (caching used to work when Actions was in early preview)

0 replies

teohhanhui · 2019-11-12T14:51:10Z

teohhanhui
Nov 12, 2019

This is really NOT a solution. The solution is to support Docker build cache (layer caching).

0 replies

FabianKoestring · 2019-11-13T11:56:55Z

FabianKoestring
Nov 13, 2019

There is a new way of doing this. https://github.com/actions/cache

But i cant find a working example of using this action with docker-compose.

0 replies

alagu · 2019-11-18T08:28:51Z

alagu
Nov 18, 2019

Yeah, I was also checking the same repo. They have built caching for generic store. Not for Docker Layer Caching.

0 replies

Pepan · 2019-11-18T13:54:41Z

Pepan
Nov 18, 2019

How to use cahing on Ruby Gems?

It always install all gems …

0 replies

Mike-E-angelo · 2019-11-24T08:16:00Z

Mike-E-angelo
Nov 24, 2019

+1

I know “+1”'s are annoying, but so are “solutions” for questions that are “solved” when they really aren’t. 😛

0 replies

moonkwoo · 2019-12-05T01:37:52Z

moonkwoo
Dec 5, 2019

For solve to this probelm, this action is proper to solve this problem. It cache the layers on registry.

0 replies

adevine · 2019-12-20T04:25:55Z

adevine
Dec 20, 2019

The way actions building works during a workflow run for actions taken from the Github Actions Marketplace seems completely bizarre to me. I mean, when you add a step that uses a marketplace action (at a specific version), the workflow automatically adds an initial step on execution to build that action, from scratch, re-running the Docker compile step every time I build. The vast majority of time spent in my workflows is for building actions from the Marketplace. To me this is completely unexpected behavior and wastes a ton of time. The whole point of the Marketplace is to allow people to reuse actions, so why do we have to build those actions from scratch every single time, with no caching?

0 replies

dougch · 2020-02-26T17:02:36Z

dougch
Feb 26, 2020

While trying to workaround the PR Secrets (https://github.community/t5/GitHub-Actions/Make-secrets-available-to-builds-of-forks/td-p/30678) problem, had the bright idea of cacheing the credentials in a container. Just a guess, but cacheing can never be enabled because of the security implications discussed in the linked community. Really wish someone would just call this out and close these types of things as ‘By Design’.

0 replies

venura9 · 2020-07-03T06:14:59Z

venura9
Jul 3, 2020

if you’re not too hung up on GitHub hosted runners, you can think about a self hosted runner. Then you have control over your runner’s cache.

https://docs.github.com/en/actions/hosting-your-own-runners/about-self-hosted-runners

0 replies

TriplEight · 2020-07-08T11:26:27Z

TriplEight
Jul 8, 2020

Could you please point out how to use the caching with the self hosted runner? Specifically with docker.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GitHub Community

Github Action Build Caching #25971

{{title}}

Replies: 16 comments

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Github Action Build Caching #25971

Replies: 16 comments

wei Aug 13, 2019 Author

wei
Aug 13, 2019
Author