Reusable Workflows, Secrets and Environments

Look at this repo

Here I show an example of the error that you may be facing, and also a solution for it which does exactly what @jenschelkopf said

I have same exact issue, this syntax literally doesn’t work.

Specifying the environment as input also doesn’t work, none of my environment secrets ever get read in this case.

@shayaantx see the repo I posted above. It will show you what you are not doing right. If you don’t understand something you can open an issue in the repo with your questions.

@jenschelkopf

As a suggestion, It would be better if reusable workflows/workflow templates allow us to do this

my-workflow-job:
      uses: AllanOricil/workflow-template-bug/.github/workflows/workflow-template-fix.yml@master
      with:
        ENVIRONMENT: myenvironment

Instead of this

my-workflow-job:
      uses: AllanOricil/workflow-template-bug/.github/workflows/workflow-template-fix.yml@master
      with:
        ENVIRONMENT: myenvironment
      secrets:
        MY_SECRET: ${{ secrets.MY_SECRET }}

It seems really weird I have to specify the secrets in the Caller file. It is also not DRY having to repeat the secrets over and over again.

I know this is the only way to make it work, because I’ve done experiments in this repo The only way to make your dynamic Github environment secret to be loaded is if you declare the secret in the caller, in the template, and you must bind your job to an environment

1 Like

Yeah, it seems the “workflow_dispatch” inputs are not passed to the called workflow. In gh-action-parameter/demo-reusing.yml at 9b1066f0d28782abb76abdf51ef701bc8f4deb60 · bahmutov/gh-action-parameter · GitHub am trying to pass an input parameter

on:
  workflow_dispatch:
    inputs:
      N:
        description: Number of parallel jobs to run
        type: number
        required: true
        default: 1

jobs:
  prepare:
    uses: bahmutov/gh-build-matrix/.github/workflows/reusable.yml@main
    with:
      n: ${{ github.event.inputs.N }}

and the run always gives me:

Failed to evaluate called workflow prepare outputs: The template is not valid. .github/workflows/demo-reusing.yml (Line: 16, Col: 10): Unexpected value '3'

Putting actual value works.

Regarding my previous post

I am getting 404 (Not found) when trying to trigger a dispatch event by running curl inside a reusable workflow .

I found out that github_token is a reserved word in the context of reusable workflows. When I changed github_token to gh_access_token, the curl command succeeded.

It would be very interesting to be able to use it like that. Because some variables are just variables and not secrets.
@jenschelkopf

Reusable workflows with in Github Actions is fundamentally broken, and it all boils down to, yet again,
the fact that engineers insist on inventing a new “programming language” with structured documents (like yaml). Gitlab (yaml) hasn’t succeeded in this. Terraform (hcl) hasn’t succeeded in this. Kubernetes (yaml) hasn’t succeeded in this. And Github is not succeeding in this either. Excuse my rant, but it is mind-numbing to see these same problems visited over and over again, by intelligent people who refuse to acknowledge that a structured, declarative document cannot give you the flexibility that you inevitably need from these systems.

Meanwhile, consumers of these broken “programming languages” sink, collectively, tens of thousands of hours into fiddling with them to do the most basic tasks that could be achieved by any programming language. The problem that this thread is addressing is solved and incredibly simple: pass some data to a reusable bit of logic. That’s it! And yet we have a thread with dozens of posts trying to accomplish this. Unbelievable!

Again, apologies for the rant, but I have been fighting with Github Actions all day, trying to pass a damn environment variable into a reusable workflow’s inputs, and it just cannot happen. Please, someone at Github, scrap Github Actions. Just give us a real (sandboxed) programming language. PLEASE. This nonsense has got to stop. Think of your users for goodness sake.

13 Likes

Just adding +1 for being able to just specify the environment as an input to the workflow and then access the secrets as usual inside the reusable workflow.

This way we can use the deployment environment UI to set the secrets, and reuse the same workflow for multiple environments, by simply passing a different environment in the input.

4 Likes

+1 as I’m not able to apply Reviews approval on specific job inside reusable workflow, “environment” parameter seems to be totally ignored, and not only this one, but “name” of job too, I tried to change the name of job inside a reusable workflow, the name is totally ignored too, it’s really a nightmare, I spent so much time to debug this…

1 Like

Thanks @AllanOricil , this saved the day for me !! Appreciate the help

Just got bit by the fact that simply moving my current workflow “as-is” in a reusable workflow to be called by other workflows in the same repository does not work, because the secrets come up empty.

I also feel like it is not the best design decision. In particular, a selling point of reusable workflows is to keep it DRY. But now, if I use X secrets in the reusable workflow, I have ot pass those X arguments on every invocation. So, copy-pasting things around is still very much alive…

3 Likes

I’m in the same boat as @Adnn.

I wanted to control the usage of a secret to my container registry by placing it as a secret within the called reusable workflow, but that doesn’t appear to be an option. So now it appears that anyone who wants to use my deployment workflow has to have said secret in their repo to pass it along.

I can’t even think of how I could make use of a 3rd party secret engine like vault without having the secret 1st come from the caller’s repo.

Has anyone found a way around this? Am I missing something and this should work?

Thanks to @jenschelkopf for her help in this thread. Here’s a working example of setting environment and sharing secrets across a calling and called workflow.

The following assumes you have an environment named development configured in your repository under the Settings tab with two secrets, FOO and BAR.

In the end, what’s happening is that there is a little back and forth between the calling and the called workflows. So, when calling the reusable workflow you pass the secrets as the secret values, even though the environment has not been declared in the calling workflow. Once inside the reusable workflow the environment is declared and the called workflow has access to the secrets for that environment in the repository of the calling workflow.

Calling workflow:

# .github/workflows/main.yml

jobs:  
  deploy:
    uses: ./.github/workflows/deploy.yml
    with:
      environment: development
    secrets:
      FOO: ${{ secrets.FOO }}
      BAR: ${{ secrets.BAR }}

So the secrets passed here don’t actually have a value yet because the environment has not been declared, but they have to be passed in at this step to give the called workflow access to the environment variables. I think this is because reusable workflows may or may not be within the same repo.

Called reusable workflow

# .github/workflows/deploy.yml

name: Reusable Deploy Script
on:
  workflow_call:
    inputs:
      environment:
        required: true
        type: string
    secrets:
      FOO:
        required: true
      BAR:
        required: true
jobs:
  deploy_step:
    environment: ${{ inputs.environment }}
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-node@v2
        with:
          node-version: '14'
      - name: npm install
        run: npm install
      - uses: jsmrcaga/action-easy-lambda-deploy@v0.0.3
        name: Update lambda code
        env:
          AWS_ACCESS_KEY_ID: ${{ secrets.FOO }}
          AWS_SECRET_ACCESS_KEY: ${{ secrets.BAR }}
        with:
          function_name: my-lambda-function
          region: us-east-1

Here the environment is declared and the reference to the secrets from the calling workflow’s repo are made available as secrets within the reusable workflow’s runtime.

2 Likes

Just got bit by the fact that simply moving my current workflow “as-is” in a reusable workflow to be called by other workflows in the same repository does not work, because the secrets come up empty.

I also feel like it is not the best design decision. In particular, a selling point of reusable workflows is to keep it DRY. But now, if I use X secrets in the reusable workflow, I have ot pass those X arguments on every invocation. So, copy-pasting things around is still very much alive…

^^ this is the crux of the problem. Please just make the repo secrets available to the called workflow jobs just like they are if they are triggered manually…

2 Likes

Just here to say that I’m super disappointed with reusable workflows.
I wanted to build a small main workflow to call a list of reusable workflows based on inputs.
It turns out I won’t be duplicating code, but I will be duplicating secret declarations between workflows in the same repository.

Folks, any help here is appreciated. Banging my head around this from past two days.

name: Caller Workflow

on:
  workflow_dispatch:

jobs:
  call-another-workflow:
    uses: Orgnization/ANOTHER-REPO-NAME/.github/workflows/yamlfilename@master
    with:
      trigger_id: ${{ github.run_id }}
    

Below is my re-usable workflow

name: Resuable Workflow
on:
  workflow_call:
    inputs:
      trigger_id:
        required: true
        type: string
   workflow_dispatch

jobs:
  some-docker-job:
    timeout-minutes: 10
    runs-on: ubuntu-latest
    outputs:
      output1: ${{ steps.runner_context.outputs.pathwd }}
    steps:
      - name: Dump runner context
        id: runner-context
        if: always()
        env: 
          RUNNER_CONTEXT: ${{ toJson(runner) }}
        run: |
          echo "$RUNNER_CONTEXT"

      - name: Checkout
        uses: actions/checkout@v2
  

On running my caller workflow, my reusable workflow is invoked all the steps are performed but on the caller workflow. “actions/checkout@v2” present in my reusable workflow in an other repo is checkout the repo of the caller workflow. Why is this behaving like this ? If anyone faced similar issues, Please guide.

This seems still weirdly broken to me; specifically when using needs context to pass the secret to a reusable workflow.

We have a GitHub app created for our CI actions. We use this action to grab a GH API token that we want to pass to the reusable workflow as a secret. We’ve added CI_APP_PEM and CI_APP_ID GH app secrets to the repo.

Here’s what everything looks like :point_down:

jobs:
  call-workflow:
    name: Call reusable workflow
    uses: foo/bar.github/workflows/reusable.yaml@main
    needs: app_ci_token
    with:
      version: ${{ github.event.inputs.version  }}
      environment: ${{ github.event.inputs.environment }}
      message: ${{ github.event.inputs.message }}
    secrets:
      token: ${{ needs.app_ci_token.outputs.app_token }}
  app_ci_token:
    name: Get CI bot token
    runs-on: ubuntu-20.04
    timeout-minutes: 10
    outputs:
      app_token: ${{ steps.get_ci_token.outputs.app_token }}
    steps:
      - name: "Get CI token"
        id: get_ci_token
        uses: machine-learning-apps/actions-app-token@master
        with:
          APP_PEM: ${{ secrets.CI_APP_PEM }}
          APP_ID: ${{ secrets.CI_APP_ID }}

Now the reusable workflow is rather simple (for brevity):

name: Reusable workflow for
on:
  workflow_call:
    inputs:
      environment:
        type: string
        description: 'Environment (stage|prod)'
        required: true
    secrets:
      token:
        required: true

jobs:
  test_reusable:
    name: Test reusable workflow
    runs-on: ubuntu-latest
    timeout-minutes: 5
    steps:
      - name: Checkout repo
        uses: actions/checkout@v2
        with:
          fetch-depth: 0
          repo: "some/repo"
          token: ${{ secrets.token }}

The problem is secrets.token is never set! It’s always an empty string. What gives? We are passing down the secret as per the docs.

The error we’re getting as a result of this is obviously:

Run actions/checkout@v2
Error: Input required and not supplied: token

I feel like the need context might be broken when pulling the output from the dependant job step, but I’m not sure – it’s just a hunch.

Are we missing something here?

It’d be great if we could have an official answer whether there are plans to improve the experience for using environment secrets in reusable workflows.

@AllanOricil’s suggestions here would improve DX and also feels more secure since we wouldn’t have to reference environment secrets in both reusable and caller workflows.

If that’s not possible, then even allowing environments: with uses: would be very helpful. Basically what @jenschelkopf wrote in Reusable Workflows, Secrets and Environments - #36 by jenschelkopf. This is currrently is not possible syntactically. Maybe this is a bug?

@jenschelkopf - is there no way to simply pass an environment context to the reusable workflow, from which the environment secrets can be accessed properly? I am having a heckuva time trying to get a simple reusable workflow to work the way it seems like it ought to.

Here’s what I’d like to do…

In the calling yaml file:

on:
  push:
    branches:
      - '*'
      - '!master'

jobs:
  quality-gate:
  uses: ./.github/workflows/quality-gates-workflow.yml

  deploy:
    needs: [ quality-gate ]
    uses: ./.github/workflows/deploy-workflow.yml
    environment: nonprod

Then in the called yaml file (./.github/workflows/deploy-workflow.yml) …

on:
  workflow_call:

jobs:
  deploy:
    name: Deploy Application
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: ./.github/workflows/composite/i-lika-deployments-cha-cha
        with:
          config: ${{ secrets.MY_CONFIG }} # environment context preset by caller
          sensitiveSecrets: ${{ secrets.MY_SENSITIVE_SECRETS }} #environment context preset by caller

The absence of this kind of behavior is creating some hokum where I have to create repo configuration with environment prefixes (eg. ${{ inputs.environment }}_MY_CONFIG & ${{ inputs.environment }}_MY_SENSITIVE_SECRETS).

While effective, this greatly undermines the ACL granularity I presume environment secrets were created for.

Is there any chance of movement on this?