How to run docker-compose inside GitHub Actions

My problem is similar to this one, but I’m not working out a solution. The error is almost identical, yet…

I’m using act with nektos/act-environments-ubuntu:18.04 full 18GB docker image to debug.

Here is docker-compose.yml:

version: '3'
services:
   app:
      build:
         context: .
      image: my_linux_python
      command: sh -c "gunicorn -b 0.0.0.0:5000 --reload --workers=1 --threads=15 application:application"
      ports:
      - 5000:5000
      working_dir: /app
      volumes:
      - ./:/app
      env_file:
      - public.env
      #- private.env
      depends_on:
      - db
   db:
      image: postgres:12-alpine
      volumes:
      - db:/var/lib/postgresql/data
      - ./:/app
      - ./db/import_schema.sql:/docker-entrypoint-initdb.d/1_import_schema.sql
      - ./db/import_demo_data.sql:/docker-entrypoint-initdb.d/2_import_demo_data.sql
      env_file:
      - public.env
      working_dir: /app
volumes:
   db:

Just in case it helps, my Dockerfile, used to create my_linux_python image:

FROM debian:buster-slim

# set work directory
WORKDIR /app

# set environment variables, to avoid pyc files and flushing buffer
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

COPY ./requirements.txt /app/requirements.txt

RUN apt-get update \
    && apt-get install --no-install-recommends -y python3-pip=18.1-5 python3-pysam=0.15.2+ds-2 \
    && pip3 --no-cache-dir install --upgrade pip \
    && pip --no-cache-dir install setuptools==49.1.0 gunicorn==20.0.4 \
    && pip --no-cache-dir install -r requirements.txt \
    && apt-get autoremove -y && apt-get autoclean -y && apt-get clean -y \
    && rm -rf /var/lib/apt/lists/*

I’m trying to run my docker-compose up inside act but it fails with the same error you’ve seen.

[2020-08-10 13:21:25 +0000] [6] [INFO] Starting gunicorn 20.0.4
[2020-08-10 13:21:25 +0000] [6] [INFO] Listening at: http://0.0.0.0:5000 (6)
[2020-08-10 13:21:25 +0000] [6] [INFO] Using worker: threads
[2020-08-10 13:21:25 +0000] [9] [INFO] Booting worker with pid: 9
[2020-08-10 13:21:25 +0000] [9] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/gunicorn/arbiter.py", line 583, in spawn_worker
    worker.init_process()
  File "/usr/local/lib/python3.7/dist-packages/gunicorn/workers/gthread.py", line 92, in init_process
    super().init_process()
  File "/usr/local/lib/python3.7/dist-packages/gunicorn/workers/base.py", line 119, in init_process
    self.load_wsgi()
  File "/usr/local/lib/python3.7/dist-packages/gunicorn/workers/base.py", line 144, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/local/lib/python3.7/dist-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/local/lib/python3.7/dist-packages/gunicorn/app/wsgiapp.py", line 49, in load
    return self.load_wsgiapp()
  File "/usr/local/lib/python3.7/dist-packages/gunicorn/app/wsgiapp.py", line 39, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/local/lib/python3.7/dist-packages/gunicorn/util.py", line 358, in import_app
    mod = importlib.import_module(module)
  File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'application'
[2020-08-10 13:21:25 +0000] [9] [INFO] Worker exiting (pid: 9)
[2020-08-10 13:21:26 +0000] [6] [INFO] Shutting down: Master
[2020-08-10 13:21:26 +0000] [6] [INFO] Reason: Worker failed to boot.

My current python-app.yml, for debugging is:

name: Python application
on:
   push:
      branches:
      - prod-live
   pull_request:
      branches:
      - prod-live
jobs:
   build:
      runs-on: ubuntu-latest
      steps:
      -  uses: actions/checkout@v2
      -  name: Run docker-compose stack
         run: docker-compose -f docker-compose.yml up -d
      -  name: Check folder
         run: pwd
      -  name: Check files
         run: ls -ltr
# if docker container app were running then... I wish I could run these...
      -  name: Lint with flake8
         run: docker-compose exec app flake8
      -  name: Check format with black
         run: docker-compose exec app black --diff --check .
      -  name: Test with PyTest
         run: docker-compose exec app python3 -m pytest 

Then, when I run act:

[Python application/build] 🚀  Start image=nektos/act-environments-ubuntu:18.04
[Python application/build]   🐳  docker run image=nektos/act-environments-ubuntu:18.04 entrypoint=["/usr/bin/tail" "-f" "/dev/null"] cmd=[]
[Python application/build]   🐳  docker cp src=/Users/alan/Programmes/phenopolis_api/. dst=/github/workspace
[Python application/build] ⭐  Run actions/checkout@v2
[Python application/build]   ✅  Success - actions/checkout@v2
[Python application/build] ⭐  Run Run docker-compose stack
Starting workspace_db_1 ... done
Starting workspace_app_1 ...
[Python application/build]   ✅  Success - Run docker-compose stack
Starting workspace_app_1 ... done
/github/workspace
[Python application/build]   ✅  Success - Check folder
[Python application/build] ⭐  Run Check files
| total 84
| -rw-r--r-- 1  502 dialout    84 Jun 25 12:46 Procfile
| -rw-r--r-- 1  502 dialout   470 Jul  5 16:06 env_vars.sh
| -rw-r--r-- 1  502 dialout  2306 Jul  8 21:22 README.md
| -rw-r--r-- 1  502 dialout 13206 Jul  8 21:26 code_setup.md
| -rw-r--r-- 1  502 dialout    31 Jul 27 09:49 pyproject.toml
| -rw-r--r-- 1  502 dialout   128 Jul 27 09:49 application.py
| -rw-r--r-- 1  502 dialout   105 Aug  3 21:50 public.env
| -rw-r--r-- 1  502 dialout   636 Aug 10 07:48 Dockerfile
| -rw-r--r-- 1  502 dialout   377 Aug 10 08:10 requirements.txt
| -rw-r--r-- 1  502 dialout   612 Aug 10 12:22 tox.ini
| -rw-r--r-- 1  502 dialout   735 Aug 10 12:52 docker-compose.yml
| -rw-r--r-- 1  502 dialout   739 Aug 10 12:59 docker-compose2.yml
| drwxr-xr-x 3 root root     4096 Aug 10 14:13 db
| drwxr-xr-x 2 root root     4096 Aug 10 14:13 response_templates
| drwxr-xr-x 4 root root     4096 Aug 10 14:13 schema
| drwxr-xr-x 2 root root     4096 Aug 10 14:13 scripts
| drwxr-xr-x 2 root root     4096 Aug 10 14:13 tests
| drwxr-xr-x 2 root root     4096 Aug 10 14:13 views
[Python application/build]   ✅  Success - Check files
[Python application/build] ⭐  Run Lint with flake8
| ERROR: No container found for app_1
[Python application/build]   ❌  Failure - Lint with flake8
Error: exit with `FAILURE`: 1

I understand that volume containing the app is not well setup, but how to change that? In the example from the similar question referred docker-compose.ci.yml and docker-compose.yml don’t mention volumes, so how can it work?
Sorry if I missed something, I’m new to GitHup action as well.

1 Like

Hi @alanwilter,

Glad to see you in Github Community Forum!

Could you please provide a sample repo for further investigation?
When you trigger the workflow in github cloud website, is the error same?

Thanks

1 Like

Thanks @weide-zhou. I don’t have permission to run merge PR in our dev-live branch so I’m asking my colleague to test my proposed python-app.yml as I did in act.

When I see the results I’ll update here.

What do you mean by sample repo. Our project in GitHub is private.

BTW, I said above I can’t merge the PRs, but I’m wondering if I could test actions on my branch, but I only see this possibility if a given branch has done a PR to our dev or prod-live branch.

If you know a way I could test in GitHub actions without this inconvenience, that would be very appreciated.

1 Like

Hi @alanwilter,

act is a third-party software, github doesn’t support workflow execution on local machine formally. Hence it’s recommended to check the workflow on cloud website.

You can create a public repository which doesn’t contain same code, just use it to reproduce the issue, you should have full access including PR merge, then we can do further investigation based on it.

Thanks.

1 Like

Dear Weide-zhou,

Indeed, once I tried my workflow in GH actions it almost worked everything out, no issues with docker.

Now I’m seeing why some pytest failed there. So far so good.

Thanks for your recommendation.

Alan

1 Like