Reusable Workflows, Secrets and Environments

In my example, secrets. PAT does exist in the referenced environment. So I’m not sure what you mean by this question. Can you rephrase?

I’m having the same issue here. I think what @pecigonzalo is referring to is that jobs.<job_id>.environment does not work when using jobs.<job_id>.uses/reusable workflows.

Example:

jobs:
  deploy:
    environment: staging
    uses: org/repo/.github/workflows/deploy.yml@main
    with:
        token: ${{ secrets.TOKEN }}  # This exists within the staging environment
The workflow is not valid. .github/workflows/deploy_caller.yml (Line: x, Col: 5): Unexpected value 'uses' .github/workflows/deploy_caller.yml (Line: y, Col: 5): Unexpected value 'with'

How can you pass a secret from a deployment environment to the reusable workflow?

5 Likes

I found a bug filed that sounds similar. I tested and it worked for me. Adding env: ${{ secrets }} in the called workflow allows my called workflow use environment secrets.

Edit: On second try, I got it working without env: ${{ secrets }}. I am using an environment called deploy where my secrets live. My calling workflow does not have environment: deploy set. However, in my calling workflow, I pass secrets to the reusable workflow (which is configured with `environment: deploy) and the workflow succeeds. It works, it’s just not the configuration I was expecting to work.

Interesting @cloin , could you post some code example ?

I’m having the same issue as @BryceBeagle mentionned, as jobs.<job_id>.environment is not usable when calling a reusable workflow.

@totogtr sure. This is an example of what I have working right now

environment: deploy has Quay username/passwords secrets.

calling workflow:

jobs:
  call-deploy-workflow:
    uses:  me/my-repo/.github/workflows/deploy.yml@main
    secrets:
      QUAY_USERNAME: ${{ secrets.QUAY_USERNAME }}
      QUAY_PASSWORD: ${{ secrets.QUAY_PASSWORD }}

reusable workflow:

# Workflow name
name: Image build

# Controls when the workflow will run
on:
  # This is a reusable workflow
  workflow_call:
    secrets:
      QUAY_PASSWORD:
        description: 'needed for registry login'
        required: false
      QUAY_USERNAME:
        description: 'needed for registry login'
        required: false

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:

  # This workflow contains a single job called "build"
  build:
    runs-on: ubuntu-latest
    environment: deploy
    steps:
      - name: Log in to quay.io
        id: registry-quay
        uses: redhat-actions/podman-login@v1
        with:
          registry: quay.io
          username: ${{ secrets.QUAY_USERNAME }}
          password: ${{ secrets.QUAY_PASSWORD }}

I was expecting to be able to do this:

Thanks, I’ll try that for environment secrets !

I’ve tried something similar for repo secrets, but instead of passing the secrets values to an action, I’d like to export them to env variables (that terraform can pick up in my case) such as

env:
  ARM_CLIENT_ID: ${{ secrets.AZURE_CLIENT_ID }}

I’m not sure if it should work but in my case it doesn’t, the env var are staying empty.

Whereas if I do

      - name: get secrets
        run: |
          echo "ARM_CLIENT_ID=${{ secrets.ARM_CLIENT_ID }}" >> $GITHUB_ENV

the env var are populated correctly.

A bit less clean way to do it in my opinion…

I am also getting this behavior (I tried multiple different combinations). The key factor here is that there are reusable values that I want to use in my env: in the “root” level. These values would have to be declared N times. Let me give you a more tangible example.

env:
  dockerRegistry: 'my_docker_registry'

jobs:
  deploy_dev:
    uses: .../.github/workflows/template.yaml@branch
    with:
      env: 'dev'
      dockerRegistry: '${{ env.dockerRegistry }}' # My docker registry is not a secret, and there is a max number of secrets that can be used per repo
  deploy_prod:
    uses: .../.github/workflows/template.yaml@branch
    with:
      env: 'prod'
      dockerRegistry: '${{ env.dockerRegistry }}'

If am not able to define custom env vars for a template, then these values have to be hardcoded in each reusable workflow call

4 Likes

Hi Jen!
I caught another little quirk in re-usable actions. I’m writing a template to use reusable workflows at scale, and within that template I’m using the GITHUB_REPOSITORY env variable (Environment variables - GitHub Docs) This var is in the format “owner/repository”. The intention is to use the var as follows:

jobs:
    call-workflow:
        uses: ${GITHUB_REPOSITORY}/.github/workflows/pull_request.yml@master

Unfortunately using this variable is causing a “invalid value workflow reference: references to workflows must be prefixed with format ‘owner/repository/’” syntax error. It would be great not to have to hardcode the repo URL when calling a workflow from within the same repository.
All the best,
Finola

also looking for a way to specify the environment that a reusable workflow will have access to the secrets for without hardcoding the environment within the workflow itself.

Something like this (from a previous comment) would be perfect

jobs:
  deploy:
    environment: staging
    uses: org/repo/.github/workflows/deploy.yml@main
    with:
        token: ${{ secrets.TOKEN }}

@matty-rose

I solved this doing it like this in the caller workflow

with:
  ENVIRONMENT: staging

Then in my reusable workflow i have:

environment: ${{ inputs.ENVIRONMENT }}

1 Like

The example above is using Bash syntax for variable expansion, but uses: isn’t interpreted by Bash. Try this instead:

jobs:
    call-workflow:
        uses: ${{ github.repository }}/.github/workflows/pull_request.yml@master

That uses the GitHub Actions expression syntax, as documented here:
https://docs.github.com/en/actions/learn-github-actions/expressions

Thanks so much for your tip! Alas, I had tried both and both produce the below error:

Heads up - you can’t use dynamic expressions to reference a reusable workflow. uses: ${GITHUB_REPOSITORY}/… and uses: ${{ github.repository }}/… are not expected to wrok.

You do need to set the environment name, but you can use an input if you don’t it to be hard-coded in the reusable workflow. Like this:

jobs:
  deploy:
    environment: ${{ inputs.environment }}
    uses: org/repo/.github/workflows/deploy.yml@main
    with:
        token: ${{ secrets.TOKEN }}

I’m having another issue with reusable workflows. I’d like to pass the working directory as an input parameter, is this expected to work ?

  blabla:
    name: "blabla"
    uses: myorg/myrepo/.github/workflows/reusable-workflow.yml@master
    with:
      directory: "my-directory" 

and on the reusable :

on:
  workflow_call:
    inputs:
      directory:
        description: The folder containing the project to build
        type: string
        required: true
[...]
jobs:
  my-job:
    name: "blablabla"
    defaults:
      run:
        working-directory: ${{ inputs.directory }}

The actual code is doing some terraform and I get an “No such file or directory” at the terraform fmt and init steps which probably indicate that I’m not in the folder containing the actual code.

Also when trying to debug that, I added a simple run: pwd step in the reusable workflow which also ends up with the same error message.
2021-11-03_16-56

Any idea how to solve this ?

1 Like

I am getting 404 (Not found) when trying to trigger a dispatch event by running curl inside a reusable workflow. The calling workflow is located in a private repo in myorg, and has a structure like this:

name: Test my reusable workflow
on:
  workflow_dispatch:
jobs:
  call-myreusableworkflow:
    uses: myorg/myrepoA/.github/workflows/myreusableworkflow.yaml@mybranch
    secrets:
      github_token: ${{ secrets.MYTOKEN }}

The reusable workflow is located in an internal repo, and has a structure like this:

name: My reusable workflow
on:
  workflow_call:
    secrets:
      github_token:
        required: true
jobs:
  handle-request:
    name: Handle request
    runs-on: ubuntu-latest
    steps:
      - name: Create dispatch event
        run: |
          DISPATCH_URL=https://api.github.com/repos/myorg/myrepo2/dispatches
          HEADER1="Authorization: token ${{ secrets.github_token }}"
          HEADER2="Content-Type: application/json"
          DATA=...
          curl --fail --location --request POST ${DISPATCH_URL} --header "${HEADER1}" --header "${HEADER2}" --data ${DATA}

The repository myrepo2, which should receive the dispatch event, is also located in an internal repo. Execution fails every time with
curl: (22) The requested URL returned error: 404
However, when I run the curl command manually in a terminal window (with the same authorization token as in the workflow), the curl succeeds. I have tested to run the workflows with a bad token value in HEADER1, and then execution fails with 401 instead of 404, so I am pretty sure that the token value in HEADER1 is correct. If anyone has a clue about what is causing this issue, and how to solve it, I would be grateful.

@jenschelkopf I’m not sure I understand this snippet you’ve provided? This is the caller of the re-usable workflow. What inputs would this be referring to? As far as I can tell, the GitHub Actions YAML parser flat-out refuses to accept environment when using uses to call a reusable workflow.

1 Like

I had the same problem and the solution worked. But it is kind weird to do it like this. Considering just the caller, I was expecting to do something like this:

    deploy-vlocdevbas:
        uses: VodafoneIS/sf-metadata/.github/workflows/deploy-template.yml@workflow_template_with_docker_image_tests
        with:
            ENVIRONMENT: vlocdevbas
    
    deploy-vlocdevpro:
        uses: VodafoneIS/sf-metadata/.github/workflows/deploy-template.yml@workflow_template_with_docker_image_tests
        with:
            ENVIRONMENT: vlocdevpro
    
    deploy-staging:
        uses: VodafoneIS/sf-metadata/.github/workflows/deploy-template.yml@workflow_template_with_docker_image_tests
        with:
            ENVIRONMENT: staging

    deploy-production:
        uses: VodafoneIS/sf-metadata/.github/workflows/deploy-template.yml@workflow_template_with_docker_image_tests
        with:
            ENVIRONMENT: prod

The reason for that is because the environment is set in the job inside the workflow template.
And because of the implemented solution I end up with this:

    deploy-vlocdevbas:
        uses: VodafoneIS/sf-metadata/.github/workflows/deploy-template.yml@workflow_template_with_docker_image_tests
        with:
            ENVIRONMENT: vlocdevbas
        secrets:
            SF_ORG_ALIAS: ${{ secrets.SF_ORG_ALIAS }}
            SF_USERNAME: ${{ secrets.SF_USERNAME }}
            SF_INSTANCE_URL: ${{ secrets.SF_INSTANCE_URL }}
            SF_CLIENT_ID: ${{ secrets.SF_CLIENT_ID }}
            SF_SERVER_KEY: ${{ secrets.SF_SERVERKEY }}
    
    deploy-vlocdevpro:
        uses: VodafoneIS/sf-metadata/.github/workflows/deploy-template.yml@workflow_template_with_docker_image_tests
        with:
            ENVIRONMENT: vlocdevpro
        secrets:
            SF_ORG_ALIAS: ${{ secrets.SF_ORG_ALIAS }}
            SF_USERNAME: ${{ secrets.SF_USERNAME }}
            SF_INSTANCE_URL: ${{ secrets.SF_INSTANCE_URL }}
            SF_CLIENT_ID: ${{ secrets.SF_CLIENT_ID }}
            SF_SERVER_KEY: ${{ secrets.SF_SERVERKEY }}
    
    deploy-staging:
        uses: VodafoneIS/sf-metadata/.github/workflows/deploy-template.yml@workflow_template_with_docker_image_tests
        with:
            ENVIRONMENT: staging
        secrets:
            SF_ORG_ALIAS: ${{ secrets.SF_ORG_ALIAS }}
            SF_USERNAME: ${{ secrets.SF_USERNAME }}
            SF_INSTANCE_URL: ${{ secrets.SF_INSTANCE_URL }}
            SF_CLIENT_ID: ${{ secrets.SF_CLIENT_ID }}
            SF_SERVER_KEY: ${{ secrets.SF_SERVERKEY }}

    deploy-production:
        uses: VodafoneIS/sf-metadata/.github/workflows/deploy-template.yml@workflow_template_with_docker_image_tests
        with:
            ENVIRONMENT: prod
        secrets:
            SF_ORG_ALIAS: ${{ secrets.SF_ORG_ALIAS }}
            SF_USERNAME: ${{ secrets.SF_USERNAME }}
            SF_INSTANCE_URL: ${{ secrets.SF_INSTANCE_URL }}
            SF_CLIENT_ID: ${{ secrets.SF_CLIENT_ID }}
            SF_SERVER_KEY: ${{ secrets.SF_SERVERKEY }}

Lots of repeated secrets over and over again. I think everyone would agree that repeating the same env variables does not seem “reusable”.

Just for what I showed, I think this feature should be revisted to improve DX.

1 Like

For anybody that find this post. Here is a repro case with the possible error a developer may reach and how to fix it.

job 1: shows the env secrets are not loaded
job 2: shows the env secrets are not loaded but the env protection rule works
job 3: shows that env secrets are loaded just copying and paste the job from the template into the caller workflow
job 4: shows how to load the env secrets. The solution is really weird but it works.

Running into same issue! I want to access global vars, and pass them along to my reusable ones but can’t!