Matrix testing stops service containers.

Hi GitHub,

I’m currently trying to test a codebase with multiple node versions and multiple databases, like so: https://github.com/JaneJeon/express-objection-starter/pull/301/files#diff-1f9e889a895c484417e82b7ba66e4e93R94

Basically, I’m using matrix w/ node.js versions as one axis and database names as the other, and I’ve “merged” all of the database configurations so that I can just swap out the database names to test with a different database.

However, as you can see here: https://github.com/JaneJeon/express-objection-starter/pull/301/checks?check_run_id=379135864, when one of the tests succeed (in this example, MariaDB w/ node 10), GitHub Actions automatically pulls down the containers (MariaDB and Redis) even though the other tests using the same database are still running, and causes them to fail.

I’ve endured so many of Github Action’s shortcomings (no YAML anchors, no way to specify a list of artifacts or even match patterns, zero documentation on how to connect to services, no documentation on what to do when you need to do ${{ foo[${{ bar }}] }}, no documentation on the fact that job.services context cannot be accessed from the job level, etc), but this is the last straw.

It honestly feels so immature and broken and poorly documented that I feel like I’m trying to get toys to do actual work… I’ve had to do so many trial and errors just to get to this point (and to figure out all of the above pain points) to the point where now I feel like the tool is actively punishing me for trying to use it for nontrivial workloads.

If anyone has a solution for matrix testing w/ service containers, great. If not, I guess I’ll just give up…

Hi @janejeon ,

I notice the latest workflow is passed on your repo, it’s successful to use node.js version as the axis and database redis as the other now, so the issue is resolved, right?

Github is always working on to support new features including yaml anchors but it’s not completed yet, sorry for the inconvenience taken. According to the syntax doc, it’s needed to set explicit value for matrix parameters, for instance: ‘node-version’ for the ‘test’ job in your latest workflow.

If you have any ideas, you’re always welcome to file a feedback ticket through github support link here: https://support.github.com/contact/feedback, thanks.