reusing/sharing/inheriting steps between jobs declarations

YAML anchors do everything I want, which includes sharing configuration not related to steps, i.e. stuff I want to share but don’t want to run like container configuration.

To prove this to myself, I decided to create a .github/workflows-src/ directory with workflow yaml files which included anchors, but only defined under the top-level anchors key. Then I created a script to pre-process that before committing.

Here’s an example (simplified) workflow yaml source file:

---
name: Integration

"on": [pull_request]

anchors:
  docker_hub_credentials: &docker_hub_credentials
    credentials:
      username: my-dockerhub-username
      password: ${{ secrets.DOCKER_HUB_PASSWORD }}

  postgres_service: &postgres_service
    postgres:
      image: mdillon/postgis:9.6
      <<: *docker_hub_credentials

  cache_gems: &cache_gems
    name: Cache Ruby Gems
    uses: actions/cache@v2
    env:
      cache-name: cache-gems
    with:
      path: vendor/bundle
      key: ${{ runner.os }}-build-${{ env.cache-name }}-gems-${{ hashFiles('Gemfile.lock') }}
      restore-keys: |
        ${{ runner.os }}-build-${{ env.cache-name }}-gems-
        ${{ runner.os }}-build-${{ env.cache-name }}-
        ${{ runner.os }}-build-
        ${{ runner.os }}-

jobs:
  lint_and_dependencies:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - *cache_gems
      - run: bundle install
      - name: Run Pronto
        run: >
          PRONTO_PULL_REQUEST_ID="${{ github.event.number }}"
          PRONTO_GITHUB_ACCESS_TOKEN="${{ github.token }}"
          bundle exec pronto run -f github_status github_pr -c origin/${{ github.base_ref }}
  run_rspec_tests:
    needs: lint_and_dependencies
    runs-on: ubuntu-latest
    services:
      <<: *postgres_service
    steps:
      - uses: actions/checkout@v2
      - *cache_gems
      - run: bundle install
      - name: Prepare database
        run: psql -h postgres -U test test_db < db/structure.sql
      - name: Run RSpec
        run: bundle exec rspec

Then I preprocess in a pre-commit hook with this script:

require "yaml"
require "json"

parsed = JSON.parse(YAML.load_file(ARGV.first).to_json)
parsed.delete("anchors")

puts YAML.dump(parsed)

Maybe this is of some use to others here, but also possibly an example of what many of us are looking for with YAML anchors for the GitHub team.

(P.S. I’m a little confused by the anchor parsing concerns as I’m not sure if I could turn off anchors in the transformation script above, but you all know your codebase better than me, of course)

5 Likes

Maybe I’m missing something but what kind of technical/security issue would there be. This is a standard feature of YAML.

Does it have to do with how expression syntax values are being expanded?

If that is the case, just parse the initial YAML first, resolving the anchors. Then validate. Then do the composites and special value expanding after that. Wouldn’t that work?

4 Likes

wink winknudge nudge – Any updates on this feature request?

1 Like

I used a shell tool yq. yq eval 'explode(.)' file.yml to dereference YAML anchors. Almost the same that @emilong suggested.

Here is SO thread with detailed description and pre-commit hook