Sharing a variable between jobs

schof · May 23, 2020, 5:39am

I have a common task that I need to perform for all job “variants” in a matrixed build. My idea was to use a separate job for that but I’m unsure how to pass a build output between two jobs.

Before needing to do a matrixed build (where we build both versions in parallel), I had been doing this all as one job. My question now is, what is the best way to share information from one job (the newly incremented verison) with the matrixed job that depends on it?

See below for some abberviated code. I haven’t tried to run it yet but I’m sure it will fail since AFAIK you can’t refernce steps from another job. I’m also aware of artifacts but we’re talking about a variable here and not some file.

TIA

jobs:
  # increment version, tag and create a release
  release:
    name: Create Release
    ...
    steps:
      - name: Bump version
        uses: ...
        with:
          repo_token: ${{ secrets.GITHUB_TOKEN }}
          major_version: 62
        id: bump_version
  build:
    name: Build APK
    needs: release
    ...
    strategy:
      matrix:
        flavor: [sandbox,live]
    steps:
      - name: Checkout
        uses: actions/checkout@v1
      - name: Set version
        run: ./client/scripts/set-version.sh
        env:
          NEW_VERSION: ${{ steps.bump_version.outputs.new_version }}
        id: set_version

yanjingzhu · May 23, 2020, 6:12am

As jobs run in different runners, those runners are in different machines. Each job runs in a fresh instance of the virtual environment specified by runs-on. It is not possible to share environment variables between machines. We don’t support to share variables between jobs.
As you metioned artifacts, we would suggest you use it . You can write the version into a file , upload it as artifact in one job and then download the artifacts in the other job , read the file content.

Please refer to: https://help.github.com/en/actions/automating-your-workflow-with-github-actions/persisting-workflow-data-using-artifacts

schof · May 23, 2020, 6:12am

I hadn’t thought of writing the variable to a file but I guess that could work. Thanks for clarifying that it’s not possible to share information any other way. I do think this could be a feature worth adding at some point. Writing to a file seems like overkill for a single variable. Thanks again.

distinctlab · May 23, 2020, 6:12am

How to explain therefore that we can define environment variables at a global level as follows (thanks):

name: Create release
 on:
  pull_request:
    types: closed
env:
  NODE_VERSION: "12.x"
  APP_VERSION: "1.0.O"
  ....
Jobs: ....

yanjingzhu · May 23, 2020, 6:12am

Hi distinctlab,

Yeah, you could set a workflow level environment variable, which could be used in all jobs. But the original poster needs set the env variable inside a job. Then the env could only be used in that job.

aroller · May 23, 2020, 6:12am

Thanks for the tip. I found Github documentation provides how to share data between jobs.

name: Share data between jobs

on: [push]

jobs:
  job_1:
    name: Add 3 and 7
    runs-on: ubuntu-latest
    steps:
      - shell: bash
        run: |
          expr 3 + 7 > math-homework.txt
      - name: Upload math result for job 1
        uses: actions/upload-artifact@v1
        with:
          name: homework
          path: math-homework.txt

  job_2:
    name: Multiply by 9
    needs: job_1
    runs-on: windows-latest
    steps:
      - name: Download math result for job 1
        uses: actions/download-artifact@v1
        with:
          name: homework
      - shell: bash
        run: |
          value=`cat homework/math-homework.txt`
          expr $value \* 9 > homework/math-homework.txt
      - name: Upload math result for job 2
        uses: actions/upload-artifact@v1
        with:
          name: homework
          path: homework/math-homework.txt

  job_3:
    name: Display results
    needs: job_2
    runs-on: macOS-latest
    steps:
      - name: Download math result for job 2
        uses: actions/download-artifact@v1
        with:
          name: homework
      - name: Print the final result
        shell: bash
        run: |
          value=`cat homework/math-homework.txt`
          echo The result is $value

distinctlab · May 23, 2020, 6:12am

Thanks for your feedback. One of the current imits of github actions is to be able to declare from a job a global (workflow)
variable that can be shared with other jobs.
I’m talking about sharing variables not data (artifacts).
Downloasd/upload artifacts actions is not practical to meet this need.

yanjingzhu · May 23, 2020, 6:12am

Yeah, you are right. Github Actions could not declare a global env in one job and use it in other jobs in a workflow. I would recommend you to share your idea in the Feedback form for GitHub Actions.

asbjornu · May 23, 2020, 6:12am

In the same way it’s possible to share variables between steps, it should be possible to share variables between jobs as well. Since a job can declare its dependence on another job, the dependee could easily have its ouput variables persisted and exposed to dependent jobs through an expansions of the existing steps.<step_id>.outputs.<output_name> syntax with jobs.<job_id>.steps.<step_id>.outputs.<output_name>. Example below:

jobs:
  version_job:
    name: Create version number
    runs-on: ubuntu-latest
    steps:
    - id: version_step
      run: echo "::set-output name=version_number::1.2.3"

  output:
    name: Output
    runs-on: ubuntu-latest
    needs: version_job
    steps:
    - run: echo "${{ jobs.version_job.steps.version_step.outputs.version_number }}"

fregante · May 23, 2020, 6:12am

@asbjornu wrote:

jobs:
version_job:
name: Create version number
runs-on: ubuntu-latest
steps:

id: version_step
run: echo “::set-output name=version_number::1.2.3”

output:
name: Output
runs-on: ubuntu-latest
needs: version_job
steps:

run: echo “${{ jobs.version_job.steps.version_step.outputs.version_number }}”

There’s no such thing as a jobs object: https://help.github.com/en/actions/reference/context-and-expression-syntax-for-github-ac

This is what happens when you run your suggested workflow:

Your workflow file was invalid: The workflow is not valid. .github/workflows/main.yml (Line: 23, Col: 12): Unrecognized named-value: ‘jobs’. Located at position 1 within expression: jobs.version_job.steps.version_step.outputs.version_number

Currently the only way to pass data between jobs is with artifacts, sadly:

https://help.github.com/en/actions/configuring-and-managing-workflows/persisting-workflow-data-using-artifacts#passing-data-between-jobs-in-a-workflow

bradymholt · May 23, 2020, 6:12am

The new jobs.<jobs_id>.outputs syntax should support this.

jobs:job1:runs-on:ubuntu-latest# Map a step output to a job outputoutputs:output1:${{steps.step1.outputs.test}}output2:${{steps.step2.outputs.test}}steps:-id:step1run:echo"::set-output name=test::hello"-id:step2run:echo"::set-output name=test::world"job2:runs-on:ubuntu-latestneeds:job1steps:-run:echo${{needs.job1.outputs.output1}}${{needs.job1.outputs.output2}}

cerna · May 23, 2020, 6:12am

Thanks for heads up, @bradymholt, this is extremely helpful.

And exactly what I needed.

cryptable · August 28, 2020, 10:02am

Probably way too late, but I use this extension to share variables between jobs.

I use it to release my artifacts into one release, so to store the release URL, which I need in another job.

...
    - name: set upload_url
      id: set_upload_url
      uses: noobly314/share-data@v1
      with:
        share-id: release_url
        mode: set
        key: url
        value: ${{ steps.create_release.outputs.upload_url }}

To get the value so I can use it again.

    - name: get upload_url
      id: get_upload_url
      uses: noobly314/share-data@v1
      with:
        share-id: release_url
        mode: get
        key: url

to access the data you do: ${{ steps.get_upload_url.outputs.data }}.