Django Docker Selenium blank page on Github Actions

Any idea why Selenium would see blank pages on GitHub Actions but see correct content and pass all tests with the same docker-compose configuration locally?

Context: New to Docker & GitHub Actions as of last week, but have almost everything working: containers deploy successfully locally & remotely, all tests pass locally including Selenium tests via docker-compose, non-Selenium tests pass on GitHub Actions. The only thing failing is Selenium tests via docker-compose on GitHub Actions.

build.yml

name: CI

on: [push]

jobs:

  build:

    runs-on: ubuntu-latest
    env:
      DATABASE_URL: postgresql://(redacted)
      DJANGO_SETTINGS_MODULE: (redacted)
      SECRET_KEY: ${{ secrets.SECRET_KEY }}
      PORT: 8000
    steps:
    - uses: actions/checkout@v2
      with:
        token: ${{ secrets.PERSONAL_ACCESS_TOKEN }}
    - name: Build the docker-compose stack
      run: docker-compose -f docker-compose-ci.yml up -d --build
    # Sleep needs to be long enough for django migrations and fixture imports to complete.
    - name: Sleep 60s
      uses: jakejarvis/wait-action@master
      with:
        time: '60s'
    - name: Check logs after sleep
      run: docker logs (projectname)_web_1
    - name: Check currently/recently run containers
      run: docker ps -a
    - name: Run a couple simple tests
      run: docker exec(projectname)_web_1 pytest billing/test_views.py
    # -T necessary for pytest to avoid "input device is not a TTY" error
    # ref: https://github.community/t5/GitHub-Actions/Workflow-using-docker-compose-running-pytest-results-I-O/td-p/56291
    - name: Run a docker-compose Selenium test
      run: docker-compose -f docker-compose-ci.yml exec -T web pytest billing/test_integration.py

docker-compose-ci.yml

version: '3.7'

services:
  web:
    build:
      context: ./app
      dockerfile: Dockerfile.ci
    volumes:
      - ./app/:/usr/src/app/
    ports:
      - 8000:8000
    environment:
      - DJANGO_SETTINGS_MODULE
      - SECRET_KEY
      - DATABASE_URL
      - SQL_HOST=db
      - SQL_PORT=5432
      - SQL_PASSWORD
    depends_on:
      - db
      - selenium-hub
  selenium-hub:
    image: selenium/standalone-chrome
    ports:
      - 4444:4444
  db:
    image: postgres:12.0-alpine
    volumes:
      - postgres_data:/var/lib/postgresql/data/
    environment:
      - POSTGRES_DB=(redacted)
      - POSTGRES_USER=(redacted)
      - POSTGRES_PASSWORD=(redacted)
volumes:
  postgres_data:

Dockerfile.ci

FROM python:3.8.0-alpine
WORKDIR /usr/src/app
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
RUN apk update \
    && apk add postgresql-dev gcc python3-dev musl-dev
RUN pip install --upgrade pip
COPY ./requirements.txt /usr/src/app/requirements.txt
RUN pip install -r requirements.txt
COPY ./entrypoint-ci.sh /usr/src/app/entrypoint-ci.sh
COPY . /usr/src/app/
ENTRYPOINT ["/usr/src/app/entrypoint-ci.sh"]
CMD tail -f /dev/null

entrypoint-ci.sh

#!/bin/sh

if [ "$DATABASE" = "postgres" ]
then
    echo "Waiting for postgres..."

    while ! nc -z $SQL_HOST $SQL_PORT; do
      sleep 0.1
    done

    echo "PostgreSQL started"
fi

echo "Starting migrations, content import, static files prep"
python manage.py flush --no-input
python manage.py makemigrations (redacted: 10 similar lines) --no-input
python manage.py migrate --no-input
python manage.py collectstatic --no-input --clear
echo "Finished executing entrypoint commands"

exec "$@"

The error occurs when Selenium tries to find the link to a Login button which should exist:

E       NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/nav/ul/div/a[1]"}

Printing page_source at the time of failure on GitHub Actions shows:

<html><head></head><body></body></html>

For reference, the URL accessed by Selenium is http://web:59687/ (web container in docker-compose-ci.yml with port dynamically assigned by Django’s StaticLiveServerTestCase)

Site content loads and tests pass locally via a docker-compose-local-ci.yml which is essentially the same running outside GitHub Actions. Any idea what’s happening?

It may be worth noting I have selenium running in docker-compose-ci.yml instead of build.yml since it allows my configuration to more closely resemble what’s already working successfully locally. I’ve tried it both places but haven’t managed to get it working either place. Appreciate any suggestions.

@anuzis,
I have helped you report this ticket to the appropriate engineering team for further investigation and evaluation. If they have any progress, I will notify you in time, and sometimes the appropriate engineers may directly reply you here.

1 Like

@anuzis,
Below is an earlier ticket about the similar question, maybe it’s helpful to you.
How to connect to a Docker container?

1 Like

Thanks, Bright Ran. Confirming I saw that before. The main difference as I understand it is removing the selenium service from docker-compose-ci.yml and putting it in the GitHub Actions build.yml instead.

docker-compose-ci.yml change:

#  selenium:
#   image: selenium/standalone-chrome
#   ports:
#     - 4444:4444

build.yml change (adding):

    services:
      selenium:
        image: selenium/standalone-chrome
        options: --health-cmd '/opt/bin/check-grid.sh'
        ports:
        - 4444:4444

Confirming I’ve just tried this change and the ‘web’ app inside the docker-compose-ci.yml can’t seem to connect to the new service at any of: localhost:4444, 0.0.0.0:4444, or selenium:4444.

E           urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='selenium', port=4444): Max retries exceeded with url: /wd/hub/session (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fdb5a4144f0>: Failed to establish a new connection: [Errno -2] Name does not resolve'))

E           urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=4444): Max retries exceeded with url: /wd/hub/session (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2cbf391d30>: Failed to establish a new connection: [Errno 111] Connection refused'))

E           urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='0.0.0.0', port=4444): Max retries exceeded with url: /wd/hub/session (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2cbf391d30>: Failed to establish a new connection: [Errno 111] Connection refused'))

It would be nice if the selenium service could run within docker-compose-ci.yml instead of build.yml since this configuration works when run locally and reduces surface area for errors, but even within build.yml I’m running into issues. Appreciate any more suggestions.

Hey, I know its been a while since you asked this question, so my apologies there. Did you ever get this working?

In your most recent update, when you switched to running selenium in a service container, it looks like the issue may be that the service container is bound to the host port 4444, but docker compose is running your containers on an app-specific network, by default. So if you try setting the network_mode of the docker-compose service to “host” it should be able to reach the selenium service, which is also on the host network

Thanks for following up with a potential fix. I didn’t get Selenium running through Github Actions and shifted attention to other areas, but am excited to see if this suggestion works as it’d be great to get everything passing via CI. I’ll followup and confirm whether this successfully resolves the issue shortly. Have a great weekend!