Accessing 'services' from another container

I’m trying to build a project which has a docker-compose.yml file, so I’m trying to test both the docker-compose and the project at the same time.

The docker-compose.yml file has a web container service for a django application and an nginx service for static files.

docker-compose.yml

version: '3.7'

services:
  web:
    command: bash -c "./bin/start.sh"
    container_name: leyfisbref_web
    env_file:
        - ./.env
    restart: unless-stopped
    build:
      context: .
      dockerfile: ./Dockerfile
    expose:
      - 80
    volumes:
      - .:/srv/leyfisbref
  nginx:
    image: nginx:stable
    container_name: leyfisbref_nginx
    ports:
      - "5000:80"
    volumes:
      - ./conf/nginx/nginx.conf:/etc/nginx/nginx.conf
      - ./static:/static_files
    restart: always
    depends_on:
      - web

Dockerfile:

FROM python:latest

# Copy default endpoint specific user settings overrides into container to specify Python path
ENV PYTHONUNBUFFERED 1

WORKDIR /srv/leyfisbref

ENV SHELL /bin/bash

# Install process tools and gettext
RUN apt-get update && apt-get -y install procps && apt-get -y install --no-install-recommends gettext

# Install python requirements
COPY requirements.txt /srv/leyfisbref
RUN pip install --disable-pip-version-check --no-cache-dir -r requirements.txt

# Clean up
RUN apt-get autoremove -y \
    && apt-get clean -y \
    && rm -rf /var/lib/apt/lists/*

# Copy files
COPY . /srv/leyfisbref

# Remove stale .pyc
RUN find /srv/leyfisbref -name "*.pyc" -delete

Database is external to the application, so I try to use a postgres service for it in the actions workflow:

pythonapp.yml:

name: Leyfisbref CI

on: [push]

jobs:
  build:
    runs-on: ubuntu-18.04
    services:
      postgres:
        image: postgres:11
        ports:
          - 5432:5432
        env:
          POSTGRES_PASSWORD: secret-password
        options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
    steps:
    - uses: actions/checkout@v1
    - name: Copy ci.env to .env
      run: cp ci.env .env
    - name: Build the docker-compose stack
      run: docker-compose up -d
    - name: Check running containers
      run: docker ps -a
    - name: Check container networks
      run: docker network ls
    - name: Check web logs
      run: docker logs leyfisbref_web
    - name: Check nginx logs
      run: docker logs leyfisbref_nginx
    - name: Run pycodestyle
      run: docker exec leyfisbref_web pycodestyle
    - name: Show django settings
      run: docker exec leyfisbref_web ./manage.py diffsettings
    - name: Run tests
      run: docker exec leyfisbref_web coverage run ./manage.py test -v 2 --failfast
    - name: Generate coverage XML
      run: docker exec leyfisbref_web coverage xml
    - name: Upload coverage to Codecov
      uses: codecov/codecov-action@v1
      with:
        token: ${{ secrets.CODECOV_TOKEN }}
        file: ./coverage.xml

In my ci.env and my project I try to use POSTGRES_HOSTNAME as ‘postgres’

When the action initialializes it creates the postgres service container as so:

Status: Downloaded newer image for postgres:11
docker.io/library/postgres:11
/usr/bin/docker create --name c4e3900ae527409195e9791e0d1bc7fd_postgres11_7016e9 --label be76db --network github_network_883d0478f5f942928f8d7fff0edc20ab --network-alias postgres --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5 -e "POSTGRES_PASSWORD=secret-password" -e GITHUB_ACTIONS=true -e CI=true postgres:11
73e4cec2c6bbdb1cdc473a344b1d8013ad1329a8fabaa66555374c70dc79a9e4
/usr/bin/docker start 73e4cec2c6bbdb1cdc473a344b1d8013ad1329a8fabaa66555374c70dc79a9e4

So it looks like ‘postgres’ is the right name (–network-alias postgres), however when my django application tries to run tests I get the following:

/usr/local/lib/python3.8/site-packages/django/db/backends/postgresql/base.py:294: RuntimeWarning: Normally Django will use a connection to the 'postgres' database to avoid running initialization queries against the production database when it's not needed (for example, when running tests). Django was unable to create a connection to the 'postgres' database and will use the first PostgreSQL database instead.
  warnings.warn(
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 220, in ensure_connection
    self.connect()
  File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 197, in connect
    self.connection = self.get_new_connection(conn_params)
  File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/postgresql/base.py", line 185, in get_new_connection
    connection = Database.connect(**conn_params)
  File "/usr/local/lib/python3.8/site-packages/psycopg2/ __init__.py", line 127, in connect
    conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: could not translate host name "postgres" to address: Name or service not known


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "./manage.py", line 21, in <module>
    main()
  File "./manage.py", line 17, in main
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.8/site-packages/django/core/management/ __init__.py", line 401, in execute_from_command_line
    utility.execute()

"could not translate host name “postgres” to address: Name or service not known"

It’s like they’re on two separate networks, the service containers and the docker-compose containers I boot up, how do I get them on the same network so they can contact each other?

Here’s the result of ‘docker ps -a’:

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
d59fb4990d07 nginx:stable "nginx -g 'daemon of…" 1 second ago Up Less than a second 0.0.0.0:5000->80/tcp leyfisbref_nginx
397bc813897e leyfisbref_web "bash -c ./bin/start…" 4 seconds ago Up Less than a second 80/tcp leyfisbref_web
f3eca62e3ec8 postgres:11 "docker-entrypoint.s…" About a minute ago Up About a minute (healthy) 0.0.0.0:5432->5432/tcp eed2374d6823450e83350ba8e0587b2e_postgres11_41817f

Here’s the result of ‘docker network ls’:

NETWORK ID NAME DRIVER SCOPE
b950d7b845ce bridge bridge local
5114e07d2325 github_network_efc32c5e86e24561933877303815839d bridge local
5c5a0d621cd8 host host local
65cf2bf7f41a leyfisbref_default bridge local
c3466626325e none null local

I know there’s a “container:” thing in Github actions, but isn’t that just for Dockerfile containers or docker hub containers? Can I specify a docker-compose container to run alongside postgres:11?

It does work if I just overload the docker-compose.yml with a file that has an extra postgres service, but I don’t really want to do that I want to test the docker-compose.yml file as is and write tests against the product as it ships or as close to it as I can get.

This whole thing is hard to google, if I look at “github actions container” I’ll get a bunch of irrelevant things. I can’t seem to define a network for the postgres service as in " networks: default: external: leyfisbref_default" or something like that, the yml would stop parsing at “networks:”. I might be able to use ${{ job.services.postgres.network }} but I don’t think docker-compose has a command line flag for overriding the network.

Naturally, as soon as I had buckled and asked for help I figured it out on my own.

I can connect the network like so:

docker network connect ${{ job.services.postgres.network }} leyfisbref_web

It will still fail in the entry point but the test command will run ok. If anyone has a better idea I’d be very grateful

Here’s my new pythonapp.yml:

name: Leyfisbref CI

on: [push]

jobs:
  build:
    runs-on: ubuntu-18.04
    services:
      postgres:
        image: postgres:11
        ports:
          - 5432:5432
        env:
          POSTGRES_PASSWORD: secret-password
        options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
    steps:
    - uses: actions/checkout@v1
    - name: Copy ci.env to .env
      run: cp ci.env .env
    - name: Build the docker-compose stack
      run: docker-compose up -d
    - name: Check running containers
      run: docker ps -a
    - name: Check container networks
      run: docker network ls
    - name: Check web logs
      run: docker logs leyfisbref_web
    - name: Check nginx logs
      run: docker logs leyfisbref_nginx
    - name: Connect leyfisbref_web to postgres service network
      run: docker network connect ${{ job.services.postgres.network }} leyfisbref_web
    - name: Run pycodestyle
      run: docker exec leyfisbref_web pycodestyle
    - name: Show django settings
      run: docker exec leyfisbref_web ./manage.py diffsettings
    - name: Run tests
      run: docker exec leyfisbref_web coverage run ./manage.py test -v 2 --failfast
    - name: Generate coverage XML
      run: docker exec leyfisbref_web coverage xml
    - name: Upload coverage to Codecov
      uses: codecov/codecov-action@v1
      with:
        token: ${{ secrets.CODECOV_TOKEN }}
        file: ./coverage.xml