[BUG] Jobs output should return a list for a matrix job

Code to Cloud GitHub Actions
#1

I am glad that the GH team implemented jobs.<jobs_id>.outputs but I found a scenario where this feature does not work as expected.

jobs:
  one:
    runs-on: ${{ matrix.os }}
    outputs:
      myvar: ${{ steps.step1.outputs.myvar }}    
    strategy:
      matrix:
        os: [ubuntu-latest, windows-latest]
    steps:
    - id: step1
      run: echo "::set-output name=myvar::${{ matrix.os }}"
  two:
    runs-on: ubuntu-latest
    needs: one
    steps:
    - run: echo '${{ needs.one.outputs.myvar }}'
    - run: echo '${{ toJson(needs.one.outputs.myvar) }}'
    - run: echo '${{ toJson(needs.one.outputs) }}'

The value of myvar is determined by the last job run, while it should be the accumulated value of all jobs in the matrix. In other words the output of a matrix job should be an array of values, not a string, one value for each execution. In the example [ubuntu-latest, windows-latest] or [windows-latest, ubuntu-latest].

9 Likes
#2

In case you wonder why one would need this, I use a matrix job to build and push docker images and a dependent job to create the “multi-arch” manifest.

1 Like
#3

Hi @giuliov,

Thank you for reaching this out!

It’s by designed. As doc mentioned below:

Job outputs are strings, and job outputs containing expressions are evaluated on the runner at the end of each job.

If you use matrix in first job, which will cause multiple checkruns for first job, the job output is determined by the last completed checkrun. The output value is overwritten by the last checkrun, it’s not accumulated.

How to share matrix between jobs
#4

I strongly disagree. First, documentation can be buggy just like code. Second, if the design is incomplete and current implementation surprise user, then design should improve and implementation as a consequence. I keep my stand that this is a bug and should be worked on at some point in time (or never).

3 Likes
#5

Hi @giuliov,

Thanks for your reply! Based on the current appearance and doc description, the job output is working as expected, but i agree there could be some improvement for the design.

According to the policy, it’s recommended to raise a feedback ticket in below link where github product manager will take a review, they will help to check.

https://support.github.com/contact/feedback?contact[category]=actions

2 Likes
#6

Has there been any developments for this feature?

1 Like
#7

I’d also like to follow up on this feature - if the output of a job is generally a string, and that job is then scattered - in the matrix use case - then the output should be an array of strings. This is common practice in workflow languages such as CWL and WDL

#8

Hey, how is this going? Any inclusion in the product roadmap? I have a similar use case for multi-arch! Are there any workarounds currently?

#9

I’ve filed a ticket through support.github on this but still would like to share my thoughts and hope it can get more traction.

I’m using Actions to build and upload assets to releases targeting different OS’s. To allow integrity checks, the workflow also hashes the assets and use it as part of the final filename.

But this makes the asset filenames nondeterministic, so I would like to generate a separate SHA256SUMS.txt instead after the build job. However, at the moment because of this issue, I cannot get all the hashes from build.

Ideally I think it’d be nice to be able to use e.g. needs.<job_id>.matrices in dependant jobs and it’s got something like:

[
  {
    "matrix": {
      "os": "ubuntu-latest"
    },
    "outputs": {
      "hash1": "foo",
      "hash2": "bar"
    }
  },
  {
    "matrix": {
      "os": "windows-latest"
    },
    "outputs": {
      "hash1": "hello",
      "hash2": "world"
    }
  }
]

This way we don’t have to break the existing context.

1 Like
#10

:+1: to the above, outputs should reflect the job’s matrix configuration, providing values for each run. My use case is enabling multi-regional deployments from regional registries.

1 Like
#11

+1 for this. Building multiple arch docker images (C++ compile) takes far too long if done in sequence, if we could do it in parallel and combine the results it would be ideal.

#12

Maybe it’s just me, but the nomenclature w.r.t. the matrix strategy seems kind of confusing. The documentation and UI both refer to each combination of matrix variables as its own “job”.

The documentation says:

A matrix allows you to create multiple jobs by performing variable substitution in a single job definition

and the UI lists them as multiple jobs accordingly. But within the dependency graph, the entire matrix is treated as a single “job”. Maybe the documentation means something else, but regardless, the heart of the matter is that there is no apparent way to query the outcome of (or depend on) an individual matrix combination.

My use case is that I, in addition to currently supported runtimes, want to target the release candidate for the upcoming runtime version in my matrix. Whether that build breaks is irrelevant to my ability to release functionality for the supported ones. It just allows me to incorporate fixes ahead of time, so that the delay between runtime release and my release hopefully isn’t that big. The only way I figure how to do this right now is to duplicate my job definition, because there is no apparent way to tell downstream which of the matrix combinations is causing a fatal failure.

EDIT: it would be great @Frederick888 if you could update us on the outcome of that ticket.

#13

I haven’t heard anything back from them unfortunately.

#14

Im trying out the same thing with

https://github.com/svishwanath-tw/talisman/master/.github/workflows/release.yml

Please suggest alternatives

#15

I suppose one could use a matrix like

matrix:
  include:
    - build: 1
      os: ubuntu-latest
    - build: 2
      os: windows-latest

and name the outputs differently:

echo "::set-output name=myvar${{ matrix.build }}::${{ matrix.os }}"

Not very convenient to access though:

- run: echo '${{ format('[{0}, {1}]', needs.one.outputs.myvar1, needs.one.outputs.myvar2) }}'

2 Likes
#16

This saved my ass. Thank you

#17

Hi @Simran-B
I’m interested in your solution. But can’t make it work at the job level.
How did you manage to pass the step dynamic vars at the job level so that you can use them in another job?

To better explain what I’m trying to achieve, I would like to expose your myvar1 and myvar2 at the job level in a dynamic way. without hardcoding their name in the job outputs section.

#18

Well I think I found some inspiration over here: How to share matrix between jobs - #2 by weide-zhou?

#19

It’s actually just one job with a matrix, i.e. it runs multiple jobs but using a single defined job, just with different parameters.

Do you want to have dynamic job output names? I don’t see why that would be necessary. It might be possible to use expressions in job.<job_id>.outputs but there could be limitations. Can you share what exactly you want to achieve with an example?

#20

Yep. That’s what I was looking for to be able to catch them in another job (let’s call it Job B).

Why in another job you might ask? Because my matrix job (Job A) uses custom docker images. And Job B uses a github action (repository-dispatch) which needs a secret. I was lazy so I didn’t bother trying to propagate my Github Secrets up to the container and comply to any other runtime requirement for the Action, I simply put the notify action in Job B with the usual ubuntu-latest runner.