Rails feature tests have stopped running javascript

Having been happily using GitHub Actions to run my Rails/Rspec/Capybara tests for many months, it has now just stopped working!

The pipelines started failing on builds on 26th Aug, around 1200 GMT.
Eg clicking on a link that performs a POST, now does a GET.
(Using https://apidock.com/rails/ActionView/Helpers/UrlHelper/link_to “method: :post”)

Any test that requires JavaScript to run fails, eg :

    Failure/Error:
      page.accept_alert 'Are you sure?' do
        click_button('Create new application')
      end

    Capybara::ModalNotFound:
      Unable to find modal dialog with Are you sure?
    # ------------------
    # --- Caused by: ---
    # Selenium::WebDriver::Error::TimeoutError:
    #   timed out after 20 seconds (no such alert
    #     (Session info: headless chrome=84.0.4147.125))

The problem can be isolated from our code by looking at previous merges to our repo.
Older merges to master that passed at the time are now failing on re-runs.
It looks as if something changed in the base build on github actions?

Our config:

name: "Tests"
on:
  push:

jobs:
  rspec:
    name: RSpec
    runs-on: ubuntu-latest
    env:
      RAILS_MASTER_KEY: ${{ secrets.RAILS_MASTER_KEY }}
      RAILS_ENV: test
      DATABASE_URL: postgres://postgres:postgres@localhost:5432/test
      REDIS_URL: redis://localhost:6379/0

    services:
      postgres:
        image: postgres:latest
        ports:
          - 5432:5432
        env:
          POSTGRES_PASSWORD: postgres
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5

      redis:
        image: redis:latest
        ports: ["6379:6379"]
        options: --entrypoint redis-server

    steps:
      - uses: actions/checkout@v1

      - name: Setup ruby
        uses: ruby/setup-ruby@v1
        with:
          bundler-cache: true

      - name: Find yarn cache location
        id: yarn-cache
        run: echo "::set-output name=dir::$(yarn cache dir)"

      - name: JS package cache
        uses: actions/cache@v1
        with:
          path: ${{ steps.yarn-cache.outputs.dir }}
          key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
          restore-keys: |
            ${{ runner.os }}-yarn-

      - name: Install packages
        run: |
          yarn install --pure-lockfile

      - name: Install postgres client
        run: sudo apt-get install libpq-dev

      - name: Install dependencies
        run: |
          gem install bundler
          bundle install --path vendor/bundle --jobs 4 --retry 3

      - name: Create database
        run: |
          bundler exec rails db:create
          bundler exec rails db:migrate

      - name: Yarn
        run: yarn test

      - name: Run tests
        run: bundler exec rspec

Thanks for your feedback.

Do you mean it worked normally before, but now it fails? If so, did you make any specific changes in your code?
Do you have a public repo demonstrating this? We can take a look to see what happened!

Yes, it was working and now it’s failing, but, as I say, fails for previously passing (unchanged) builds.

As I say above, previous merges to master that passed at the time are now failing on re-runs.
In case that is not clear, a build that passed fine a week ago, which then we merged because it passed CI, now fails. That is, if I open that commit and see it shows a green tick stating the CI had passed, I open that and re-run the tests, they now fail. Thus meaning that it is not something under our control but something that GitHub have done to the image.

Unfortunately not this is a private repo, but unless you are an employee of GitHub, I don’t see what looking at the repo will show you, because as I say, no changes in our repo have caused this change.

Hi @IanVaughan
Since it is a private repo, could you please create another public demo repo which can reproduce this issue so that we can investigate further.