How to have atomic releases

I have a job that uses a matrix strategy to build for the various platforms. But these turn all into separate build instances. They all build their own artifacts, but how can I get access to those build artifacts in job that runs after?

I only want to create a release when all builds artifacts (including docker images) got build successfully. Otherwise I might end up with a half broken release.

Or am I approaching this the wrong way?

Could you just use the API to retrieve the tag (before it’s deemed a release) and check that it has all the expected artifacts?

It feels a little silly to check, then download the artifacts, then put them into docker images. But if there is no better way it certainly could be a work around.

@tcurdt ,

Maybe you can reference to the following demo to have a try in your project:

jobs:
  job1:
    name: Build
    . . .
    strategy:
      matrix:
        platforms: [platform-1, platform-2, ...]
    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Build
        # Execute related action or command lines in this steps 
        # to build your project and generate the artifact files

      - name: Copy artifacts
        run: |
          mkdir -p MyArtifacts
          cp -r path/to/artifacts/* MyArtifacts
# Copy the artifact files from project directory into a special folder (e.g. 'MyArtifacts'),
# so that it is more convenient for uploading the artifact in the subsequent step.

      - name: Upload artifact
        uses: actions/upload-artifact@v2
        with:
          name: artifact-name-${{ matrix.platforms }}
          path: MyArtifacts
# Use the 'upload-artifact' action to upload the artifact files from 'MyArtifacts' 
# with the name you specify. You can set the artifact name with the corresponding platform 
# information so that it is more convenient for distinguishing them in the subsequent job.

  job2:
    name: Create release
    needs: [job1]
    outputs:
      upload_url: ${{ steps.create_release.outputs.upload_url }}
    runs-on: ubuntu-latest
    steps:

      - name: Create Release
        id: create_release
        uses: actions/create-release@v1
        with:
          tag_name: <tag_name>
          release_name: <release_name>
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Use the 'create-release' action to create a new release 
# with the specified tag and release name

job3:
  name: Upload Release Assets
    needs: [job1, job2]
    runs-on: ubuntu-latest
    . . .
    strategy:
      matrix:
        platforms: [platform-1, platform-2, ...]
    steps:
      - name: Download artifact
        uses: actions/download-artifact@v2
        with:
          name: artifact-name-${{ matrix.platforms }}
          path: MyAsset
# Use the 'download-artifact' action to download 
# the artifacts (uploaded in job1) into a special folder (e.g. 'MyAsset').

      - name: Zip artifact files
        run: |
          cd MyAsset
          zip -r asset-name-${{ matrix.platforms }}-version.zip *
# Compress the artifact files as a ZIP file with the name you specify.
# you can set the name with the corresponding platform information 
# and the version of the release (generally is the tag name).

      - name: Upload Asset
        uses: actions/upload-release-asset@v1
        with:
          upload_url: ${{ needs.job2.outputs.upload_url }}
          asset_path: MyAsset/asset-name-${{ matrix.platforms }}-version.zip
          asset_name: asset-name-${{ matrix.platforms }}-version.zip
          asset_content_type: application/zip
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Use the 'upload-release-asset' action to upload the 
# ZIP file as the asset of the release created in job2.

Below is the result of an example from my project.
example

If you don’t want to upload the artifacts as release assets, omit job3.
Hope this can help you.

1 Like

@brightran Thanks!

It’s a shame there is no way other to share the artifacts between the jobs (the cache maybe?), or to have a single definition of the build matrix. But this example does give a few ideas. Very helpful.

I guess the main question is whether that’s the suggested way of doing this. Or how other people are handling e.g. aborted releases.

@tcurdt ,

Normally, we can use artifacts to share data between jobs in a same workflow run. Just like as the example I shared above, we can use the ‘upload-artifact’ action and ‘download-artifact’ action to pass and share data between jobs.
If you want to share data between workflow runs or download artifacts from a previous workflow run, you may need to use the “Artifacts” API.
More details about artifacts, you can see “Persisting workflow data using artifacts”.

About cache, it generally is used to share the often reused dependencies between jobs or workflow runs, and typically the dependencies don’t change often between jobs or workflow runs.
Sure, you also can use cache to share build artifacts between jobs or workflow runs.
More details about cache, you can see “Caching dependencies to speed up workflows”.