`needs` based on `matrix`

I build my SW using a matrix, let’s say:

    os: [windows, linux]
    configuration: [Release, Debug]
    architecture: [amd64, arm64]

Then, when I want to test the SW I use another matrix:

    os: [windows, linux]
    configuration: [Release, Debug]
    architecture: [amd64, arm64]
    tests: [foo, bar, baz, ....]

All the “tests” are big collections of tests that must be run in parallel (only limited by free runners) since they are time-consuming.

Testing must wait for the corresponding build to be done so I would like to do:

needs: build-${{ matrix.os }}-${{ matrix.configuration }}-${{ matrix.architecture }}

However, this results in an error, all I can do is:

needs: build

This puts in an unnecessary barrier in the pipeline that hits me twice. First, all testing must wait until all building is done. Then, once all building is done all tests are fired at once resulting in unnecessary queues.

Other job properties like runs-on and steps can depend on a matrix, so this seems like an oversight that should be fixed.

  • Are there any known workarounds for this? Matrix inside matrix or code composition would probably work, but both those features are also missing.
  • Is this a feature to be expected soon?
  • Is there any bug report I can subscribe to?
1 Like

@nedrebo ,

There are few points you need to understand:

  1. The value of the ‘needs’ should be a job ID or an array of job IDs. The string or each element in the array will be directly considered as a job ID, any variables or expressions contained in the string will not be recognized as their corresponding values.

  2. In a workflow, each job has an unique ID that you need to explicitly set it in the workflow file, and the ID can’t be changed dynamically during the workflow run. All the jobs generated from a job matrix share a same job ID.

  3. The usage of the matrix context:

    • You can use the matrix context to set the value of the ‘name’, ‘runs-on’ and ‘with’ keys, or in the step’s if conditional, or in the run step’s command lines.

    • You can’t use the matrix context to set the value if the ‘id’ (job ID, step ID) and ‘needs’ keys, and also can’t be used in the job’s if conditional.

If you really need the feature that generate a job ID for each job in the matrix, and a job can depend (needs) a specified job in the matrix, I recommend that you can directly report a feature request here.
That will allow you to directly interact with the appropriate engineering team, and make it more convenient for the engineering team to collect and categorize your suggestions.