Support global environment variables

It would be great if variables from the job matrix are made available as environment global variables. This would simplify setting the compiler when working with C/C++ as we could do the following:

matrix:
  CC: [gcc-6, gcc-7, ...]

which is then automatically picked up by (most) build systems. There are other environment variables that are automatically picked up by build systems but currently have to be manually passed to the build system which results in an unnecessarily large CI workflow.

An option to transfer environment variables between steps would also be really useful. Specifically, when compiling C++ on Windows, we need to run a batch script that sets a whole bunch of environment variables required for using the MSVC compiler. However, when we’re using the CMake cross platform build system we don’t want to duplicate the CMake invocation for Windows and Linux. Currently, we’re forced to do so as environment variables cannot be transferred between steps.

Currently:

- name: Configure
  if: startsWith(matrix.name, 'linux')
  run: |
    export PATH=/usr/local/bin:$PATH
    cmake -B build -S . -DCMAKE_BUILD_TYPE=Debug ...

- name: Configure
  if: startsWith(matrix.name, 'windows')
  run: |
    set PATH=C:\ProgramData\scoop\shims;%PATH%
    call "${{ matrix.vcvarsall }}" x64 # Sets a whole bunch of environment variables used by the next call to cmake.
    cmake -B build -S . -DCMAKE_BUILD_TYPE=Debug ... # Same as above (duplicated)

What I would like it to be:

- name: Configure (Linux)
  if: startsWith(matrix.name, 'linux')
  run: export PATH=/usr/local/bin:$PATH

- name: Configure (Windows)
  if: startsWith(matrix.name, 'windows')
  run: |
    set PATH=C:\ProgramData\scoop\shims;%PATH%
    call "${{ matrix.vcvarsall }}" x64

- name: Configure
  run: cmake -B build -S . -DCMAKE_BUILD_TYPE=Debug ...

This is currently not possible as the environment variables set in the previous step are not available in the next step where CMake is called.

To conclude, adding global environment variables would vastly simplify compiling C/C++ projects using Github Actions CI. I assume that there are also other scenarios that would benefit immensely from global environment variables.

Regards,

Daan

6 Likes

We have an update coming to the language to enable you to sepcify an ENV: key at the job or workflow level.  With that you will be able to do the following:

jobs:
  build:
    strategy:
      matrix:
        compiler: [gcc-6, gcc-7]
    env:
      CC: ${{ matrix.compiler }}
    steps:
      .....

and all steps will have the CC environmnt variable defined when they are run.  I don’t want to arbitrarily add all variables to the environment as that can result in a bloated environment and would likely require some way to negate that behavior. 

Update we now support workflow and job level environment variables https://github.blog/changelog/2019-10-01-github-actions-new-workflow-syntax-features/

16 Likes

While this would definitely be an improvement, this approach is limiting when we only want to set an environment variable in some parts of the build matrix.

To give an example, in reproc’s Windows CI (https://github.com/DaanDeMeyer/reproc/blob/master/azure-pipelines.yml), I only set the CFLAGS and CXXFLAGS in a few different configurations but not all of them. More generally, some programs (such as CMake) use a default value if an environment variable is not set. With the outlined approach, we would be required to always specify the default value for the environment variable manually which adds unnecessary boilerplate and also breaks down when the default value changes between different versions of the software. We have to specify the default value manually since an empty but set environment variable might behave differently than a missing environment variable.

To provide another data point, Azure Pipelines does take the approach of making all job variables available as global environment variables. For reproc’s Windows CI specifically, this approach has been helpful and reduced boilerplate. It might be a good idea to check with the Azure Pipelines team to see what their overall experience with this approach has been.

I do agree that if this approach is taken it should probably not be enabled by default to avoid hard to debug issues. However, it would be nice to have options to make job variables available as environment variables and to transfer environment variables between different job steps.

1 Like

To give a more complete example, we can look at reproc’s CI which I’ve finished porting to Github Actions: https://github.com/DaanDeMeyer/reproc/blob/master/.github/workflows/main.yml

Once https://github.community/t5/GitHub-Actions/YAML-folded-multiline-strings-interpreted-as-multiple-commands/m-p/30876#M593 is fixed, The Linux/MacOS and Windows Configure steps will contain almost exactly the same cmake invocation but they still cannot be combined into a single step because the Windows Configure step needs to execute two extra commands that modify the environment which cannot be put into a separate step because each step resets the environment.

Similarly, the Build and Test steps have exactly the same problem where they are almost identical between Linux/MacOS and Windows but still have to be duplicated because we cannot put the extra commands that modify the environment on Windows in a separate step.

Question.  Would this allow the envvars at the job level to be set based on dynamic values?  My use case involves some logic at the beginning to build up a custom docker image:tag (i know there are built-in actions for this but i need more control) and then I want to reference the image value in subsequent steps to build, push, deploy the image.

run: |
          BRANCH=$(echo $GITHUB_REF | cut -d "/" -f 3)
          SHA_SHORT=$(echo $GITHUB_SHA | head -c7)
          echo "$REPO:$VERSION-$BRANCH.$SHA_SHORT" >> ./image

The challenge is that subsequent actions that need to reference files need seem to need shell expansion, so you can’t use many standard docker image actions.

- name: Build image
        uses: docker://quay.io/turner/fargate-cicd
        with:
          args: docker build -t $(cat image)

Thoughts?

Check https://help.github.com/en/articles/development-tools-for-github-actions#set-an-environment-variable-set-env.

It *might* work to set an environment variable for subsequent steps by echoing a string like

echo ::set-env name=action_state::yellow

2 Likes

ENV vars at workflow level are already working.

I have this in my workflow:

env:
  IMAGE_NAME: go-api-sample:${{github.sha}}

It would be nice to have some kind of Changelog for all the changes being made. It´s hard to know what bugs are being fixed and new features added.

2 Likes

You can set global environment variables like this.  It perhaps isn’t the neatest way but it works :slight_smile:

- name: Set environment variables
        shell: bash
        run: |
          echo '::set-env name=GOPATH::${{ runner.workspace }}'
          echo '::add-path::${{ runner.workspace }}/bin'
          echo '::set-env name=GO111MODULE::${{ matrix.modules }}'
          echo '::set-env name=GOTAGS::${{ matrix.gotags }}'
          echo '::set-env name=BUILD_FLAGS::${{ matrix.build_flags }}'
          if [["${{ matrix.goarch }}" != ""]]; then echo '::set-env name=GOARCH::${{ matrix.goarch }}' ; fi
          if [["${{ matrix.cgo }}" != ""]]; then echo '::set-env name=CGO_ENABLED::${{ matrix.cgo }}' ; fi
3 Likes

Its official now the support for Environment variables at job and workflow level. :wink:

Here is the announcement.

2 Likes

@chrispat Great news! Can we add environment variables at workflow level?

For example using 

echo ::set-env

@botellaa 

set-env will set environment variables for the remainder of the steps in the job but not for other jobs.  

Is there any way to set a variable at workflow level?

@botellaa wrote:
Is there any way to set a variable at workflow level?

Yes, simply set an env key at the top workflow level.

Also see https://help.github.com/en/articles/workflow-syntax-for-github-actions#env for further reference.

Ok, sorry I did not explain clearly my issue. I would like to add a variable from an action and get this variable from another job. The goal is to condition another job with it. I am hopping that this new environment variable at workflow level could be used for this.

1 Like

@chrispatIt seems neither workflow nor job level env variables are available to services, is this by design, and if so what is the reasoning behind this? A clear use case that I’m looking at right now is spinning up a database for tests, and having to specify credentials multiple times.

We have tried to be careful about how many things we automatically pass into services as different environment variables and mounts can have an impact that is not easy to debug.  Right now services do not get the entire ENV but we are working on adding a way to access the workflow env through a new ENV context that you can use in expressions i.e. ${{ env.VAR_NAME }}.  Once this is in place you would be able to define the value once and just pass it in where you need it.

2 Likes

@chrispatThat sounds like a good trade off, and I also understand and agree with your reasoning around env and services. Any idea on when something like that might be available? (ballpark if it’s in a few weeks or not until next year is fine with me).

It should be out before GA in November.

[A screenshot of your package]

@botellaa Have you been able to achieve the desired goal?