Monorepo - actions on changed folders only

Code to Cloud GitHub Actions
#1

I have a mono repo containing many projects and a variable depth folder structure.

I have a number of tasks that need to run once only withing a folder that contains code changes this should run on all changes committed since the last push

Example test code looks like below:

name: code-analysis-doc-and-lint
on:
  push:
    branches:
    - '**'
    paths:
    - '**/**'
jobs:
  scan-files-for-changes:
    runs-on: ubuntu-latest
    outputs:
      matrix: ${{ steps.check-for-changes.outputs.folders }}
    steps:
      - name: checkout
        uses: actions/checkout@v2
        with:
          fetch-depth: 0
      - name: check-for-changes
        id: code-scan
        run: |
          echo "::set-output name=folders::$(git diff-tree --diff-filter=d --no-commit-id --name-only -r ${{ github.sha }}| xargs -L1 dirname | uniq )"
          echo ${{steps.check-for-changes.outputs.folders}}
      - name: tf_docs
        uses: Dirrk/terraform-docs@master
        with:
          tf_docs_working_dir: ${{steps.check-for-changes.outputs.folders}}
          tf_docs_working_file: 'usage.md'
      - name: Run Checkov
        id: checkov
        uses: bridgecrewio/checkov-action@master
        with:
          directory: ${{steps.check-for-changes.outputs.folders}}
          skip_check: CKV_AZURE_1,CKV_AZURE_41
          quiet: 'true'
          framework: all

However whilst the "set-output name=folders::$(git diff-tree… " does show a unique list of folders that contain code changes. I don’t seem to be able to pass that list through as outputs to other jobs.

echo ${{steps.check-for-changes.outputs.folders}} returns an empty folder.

I have also tried sending the output to an environment variable but I get an error saying the format is not supported.

Any advice is appreciated

#2

To reference an output you need to use the id, not the name, so the output should be steps.code-scan.outputs.folders.

#3

Thanks,
That was useful as it seems to get something but still not what I was expecting
I’ve made some test code to use that

name: code-analysis-doc-and-lint
on:
  push:
    branches:
    - '**'
    paths:
    - '**/**'
jobs:
  scan-files-for-changes:
    runs-on: ubuntu-latest
    outputs:
      matrix: ${{ steps.code-scan.outputs.folders }}
    steps:
      - name: checkout
        uses: actions/checkout@v2
        with:
          fetch-depth: 0
      - name: check-for-changes
        id: code-scan
        run: |
          echo "::set-output name=folders::$(git diff-tree --diff-filter=d --no-commit-id --name-only -r ${{ github.sha }}| xargs -L1 dirname | uniq )" 
          echo "This should now show the list of folders" 
          echo ${{steps.code-scan.outputs.folders}}
      - name: list-all
        id: sanity-check
        run: |
          for folder in ${{steps.code-scan.outputs.folders}}; do
            echo $folder
          done

The git diff shows all the changed folders

However
echo ${{steps.code-scan.outputs.folders}} still shows nothing

and more confusingly

      - name: list-all
        id: sanity-check
        run: |
          for folder in ${{steps.code-scan.outputs.folders}}; do
            echo $folder
          done

only runs once with ${{steps.code-scan.outputs.folders}}; expanding out into a single changed folder (one in the middle of the list)

#4

That’s to be expected because it’s in the same step that’s setting the output. Try moving it to a separate step, I hope the result will give some hint why the for loop is behaving weirdly.