What's the recommended way to pass a Docker image to the next job in a workflow

GitHub Actions
#1

In my case I have a workflow with four jobs:

  1. Job 1 builds a Docker image and then pushes to it Dockerhub.
  2. Job 2 runs unit tests.
  3. Job 3 runs security scans.
  4. Job 4 waits for the above 3 jobs to finish and then pulls the built Docker image from Dockerhub and runs some smoke tests on it.

Pulling the Docker image in job 4 wastes a lot of time. Is there a recommended way for passing Docker images between jobs in a workflow?

Ideally we’d like to combine job 1 and 4 for the best performance, but because of job 2 and 3 being a dependency of job 4, we can’t do that.

14 Likes
#2

As different jobs use different runners, so we can’t pass docker image between jobs.

Is it possible to share dockerfile between job1 and job4 in your scenario, and then build it in job4 again to generate docker image instead of pulling it from docker hub?  

You can use the upload-artifact and download-artifact actions to share data between jobs in a workflow.

https://help.github.com/en/actions/automating-your-workflow-with-github-actions/persisting-workflow-data-using-artifacts#passing-data-between-jobs-in-a-workflow

#3

Pulling the image is faster than re-building the image and using the upload-artifact/download-artifact actions. I’ve tried both.

1 Like
#4

Github provided Github Package Registry which could hold your docker image like docker hub registry. If you are using public repo, it is free. But there are some limits for private repo.

GPR.png
GPR.png1155×781 39.7 KB

If you’d like to take it into consideration, you could follow this document to push to and pull from Github Package Registry.

https://help.github.com/en/github/managing-packages-with-github-packages/configuring-docker-for-use-with-github-packages#further-reading

#5

@yanjingzhu that still has the same problem I mentioned in the original post.

#6

Sorry to tell that there are no other methods to pass Docker image to next job in a workflow. 

#7

I think this is a great feature request. it is a common pattern to build a docker image then run tests in that docker image as a seperate and distinct job step. Other git hosting services have this feature in thier CI/CD piplines.

25 Likes
#8

Can you use steps instead of jobs? Steps can reuse artifacts from previous steps and it sounds like the OP doesnt have any need for running jobs in parallel, its a sequential workflow.

#9

it sounds like the OP doesnt have any need for running jobs in parallel, its a sequential workflow.

It’s not a sequential workflow since jobs 1, 2, and 3 run in parallel and job 4 waits for all of them to pass.

5 Likes
#10

Yes we should be able to build an image and then use it in subsequent steps, that’s vital to automating CI workflow.

7 Likes
#11

+1

Need this

7 Likes
#12

Kind of shocked that this doesn’t exist. I mean seriously? This is kind of a hallmark of any good CI/CD system. Why would you go and develop GitHub actions in a box like this without borrowing from the many examples out there of modern CI/CD systems?

2 Likes
#13

Yes this feature is critical in a CI framework, some support for it would be very helpful!