I have a template workflow yml file with 2 jobs, each with the same matrix strategy. The second job depends on the first with needs. I’d like to know if there’s a way to run each value in the matrix async so that the second job doesn’t require all matrix values to complete in the first job before moving on to the second one?

Example main.yml:

on: [push]
jobs:

ssql:

runs-on: ubuntu-latest

strategy:

matrix:

FRUIT: ["apple", "pear"]

name: Get data

steps:

- name: Checkout cum-rev repo

uses: actions/checkout@v2 # Defaults to current repo - check out current repo

- name: Checkout ds-ssql-gh-action

uses: actions/checkout@v2

with:

repository: ourorg/ds-ssql-gh-action

token: ${{ secrets.WORKFLOW_TOKEN }}

ref: main

path: './ds-ssql-gh-action'

- name: Run query and save data

uses: ./ds-ssql-gh-action

- name: Upload data as artifact

uses: actions/upload-artifact@v2

with:

name: ${{ matrix.FRUIT }}-data

path: output_data/data.csv
Rtrain:

runs-on: ubuntu-latest

name: Train model

needs: ssql

strategy:

matrix:

FRUIT: ["apple", "pear"]

steps:

- name: checkout current repo

uses: actions/checkout@v2 # Defaults to current repo - check out current repo

- name: Retreive data from ssql job

uses: actions/download-artifact@v2

with:

name: ${{ matrix.FRUIT }}-data

- name: Just check csv file is there

run: ls -l

- name: Fit model and save as RDS

uses: ./Rtrain # uses an action in this directory

- name: Check the model is there

run: ls -l

- name: Upload model as artifact

uses: actions/upload-artifact@v2

with:

name: ${{ matrix.FRUIT }}-Model

path: ${{ matrix.FRUIT }}-Model.RDS

Where I use needs in the second job, I really mean ‘needs the ssql job for the corresponding fruit, not all jobs in ssql to complete first’.

Suppose the ssql job for ‘apple’ takes 3 times as long as for ‘pear’. I would like the next job, Rtrain, to proceed once the ssql job for ‘pear’ is done without waiting for the ‘apple’ one to complete.

Is this possible?