Help
cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Pilot Lvl 1
Message 1 of 9

GitHub Actions: Using non-fixed names in upload-artifact

Solved! Go to Solution.

Hi,

 

How could I upload an artifact the name of which is not known when writing the YAML file?

 

More precisely, the typical use case is a CI which builds the binary package of an application. The name of the package file contains the version number of the application. This version number has been found somewhere or have been computed somehow by the build procedure. At the end of the build procedure, we know this version number and we know the package file name. But how can we specify the file name to upload-artifact?

 

When hard-coding the names in the YAML file, this would be something like:

jobs:
windows-msvc:
name: Windows with VC++
runs-on: windows-latest
steps:
- uses: actions/checkout@master
- name: build
run: build-app.ps1
- name: upload uses: actions/upload-artifact@master with: name: app version 1.3.45 path: app-1.3.45.exe

But, of course, this is not appropriate because, next time, the version number will be 1.3.46 or 1.4.0 or 2.0, who knows?

 

It could be something like:

    - name: upload
      uses: actions/upload-artifact@master
      with:
        name: app version ${{ env.VERSION }}
        path: app-${{ env.VERSION }}.exe

But, where do we set the environment variable VERSION? If set in a previous step, the environment is not propagated to the next step. I do not see any possibility to run code in the same step as the upload ("run" and "uses" cannot be used in the same step).

 

I hope you get the idea. Any solution would be appreciated.

Thanks

-Thierry

8 Replies
Copilot Lvl 3
Message 2 of 9

Re: GitHub Actions: Using non-fixed names in upload-artifact

Hey, @lelegard! Good question.  If the step creating the version and the step needed the version are both run in the same job, you can set the environment variable at the job level.

This worked for me:

name: Main
on: [push]
jobs:
  windows-msvc:
    env:
      VERSION: 1.3.45
    name: Windows with VC++
    runs-on: windows-latest
    steps:
    - uses: actions/checkout@master
    - name: build
      run: ./build-app.ps1
    - name: upload
      uses: actions/upload-artifact@master
      with:
        name: app-version-${{ env.VERSION }}
        path: app-${{ env.VERSION }}.exe

This way the build and  upload steps both get the same values for env.VERSION.  Your script will read it from the environment.  

 

To test I used this as `build-app.ps1`:

$file = "app-" + $Env:VERSION + ".exe"
echo $file | Out-File $file

 

If the version info is needed in a different job, you will need to create an artifact for the value and upload it in the first job, then download it and use it in the next job.

 

Hope this helps!  Let us know how you fare.

Highlighted
Pilot Lvl 1
Message 3 of 9

Re: GitHub Actions: Using non-fixed names in upload-artifact

Hi @managedkaos,

Thank you for you answer. In my case, specifying the version number in an "env:" section of the YAML file is not an option. The YAML file is not supposed to change. I want automation, not manual modification of a file any time something has changed. Typically, the version number is in a header file (.h, this is a C++ project). The minor version is automatically incremented through git hooks each time a commit is processed. The build script can read the version number but I do not want any manual operation.

 

So, in the job, one step builds the application and "discovers" the version number. The artifact file contains the version number in its name (eg. app-1.3.45.exe). In a later step in the same job, I want to upload this artifact. So, the version number must come from a previous step in the job but it shall not need any manual modification of the YAML file.

 

In that case, the artifact is meant to be publicly downloaded by users. It is not used by another job. This is why keeping the version number in the artifact name is important for the user.

Highlighted
Copilot Lvl 3
Message 4 of 9

Re: GitHub Actions: Using non-fixed names in upload-artifact

I think the piece you are missing is that steps in a job can set environment variables for use by later steps in the job by echoing specially-formatted strings, as documented here. I do that all over the place in my own workflow, for reasons just like you are describing. The YAML file doesn’t need to hard-code all environment variables, earlier steps can set them up based on whatever process you need.

Highlighted
Copilot Lvl 3
Message 5 of 9

Re: GitHub Actions: Using non-fixed names in upload-artifact

Agree!  The workflow should take care of the settings automatically and not require manual intervention.

 

If the version data is in a file, I think the key will be reading it from a file and using it in your build steps vs using an environment variable.

 

I see one suggestiont that mentions how a step can update an environment variable that can be used in a future step.  However, that won't work in the case of using the artifact upload action since the variable would need to be in the configuration at the Workflow level.

 

Instead, you'll likely need to do two things:

  1. Update your build script to read the header file into a variable and use that accordingly to create the executable with the correct version name.  I am assuming the build script writes the file to a known location that will always be the same.
  2. Hard code the output location in your artifact upload step.  That way, the exe with the version naming will always be picked up from the correct location and you won't have to worry about configuring the workflow to name the exe.  Note that in the artifact upload action, you can specify a path, not necessarily a file name.  If you specify a directory, the action picks up all the files in that directory.

 

Here's an example I got working:

 

version.txt; Version file (my stand in for your header file):

1.3.45

build-app.ps1; Build script:

New-Item -ItemType directory -Path ./output
$version = Get-Content ./version.txt -Raw 
$version = $version -replace "`n|`r",""
$file = "app-" + $version + ".exe"
echo $version | Out-File ./output/$file

Workflow:

name: Main
on: [push]
jobs:
  windows-msvc:
    name: Windows with VC++
    runs-on: windows-latest
    steps:
    - uses: actions/checkout@master
    - name: build
      run: ./build-app.ps1
    - name: upload
      uses: actions/upload-artifact@master
      with:
        name: app
        path: ./output

 

Output:

2020-01-12T18:56:01.9520969Z ##[group]Run actions/upload-artifact@master
2020-01-12T18:56:01.9521203Z with:
2020-01-12T18:56:01.9521294Z   name: app
2020-01-12T18:56:01.9521463Z   path: ./output
2020-01-12T18:56:01.9521587Z ##[endgroup]
2020-01-12T18:56:02.2515502Z Uploading artifact 'app' from 'd:\a\read-artifact-version-from-file\read-artifact-version-from-file\output' for run #9
2020-01-12T18:56:02.3437290Z Uploading 1 files
2020-01-12T18:56:02.6906586Z File upload complete.
2020-01-12T18:56:02.6911850Z Uploaded '8' bytes from 'd:\a\read-artifact-version-from-file\read-artifact-version-from-file\output' to server

On another note, you may also consider creating a release along with uploading an artifact.  Artifacts are only made avaialable for 90 days and the amount of storage avialable for artifacts is limited by the type of GitHub account you have.  Also, you might be able to share informationf or downloading releases more programatically vs sharing and/or using links to download artifacts as there'e no API at the moment to support that; you and your uses will have to go directly to the run that created the artifact and download it from there or you'll have to manually track down the link to the artifact and share that.

 

Please keep the discussion going; let us know how things work out.

 

-MK

Highlighted
Copilot Lvl 3
Message 6 of 9

Re: GitHub Actions: Using non-fixed names in upload-artifact

Actually, @managedkaos using the mechanism I mentioned whereby a step can set an environment variable for use in subsequent steps and jobs does work even when you are using actions like the artifact upload action, because you can use the workflow expression syntax to interpolate the environment variables into the action configuration, and the values set by your previous step will, in fact, be used. I do exactly that in my own workflow. Here is where I set some environment variables in an earlier step, and here is where I use those values in a later step with an upload action.

 

It’s really an amazingly powerful and useful framework GitHub has set up for us here! :)

Highlighted
Solution
Copilot Lvl 3
Message 7 of 9

Re: GitHub Actions: Using non-fixed names in upload-artifact

Yep, so it is @brunchboy!  That's pretty cool. :D

I've validated by making the following changes:

  1. Set the global VERSION environment variable in the build step after reading it from the file.
  2. Build step creates the file at the root level with the VERSION vs inside a directory.
  3. Update the workflow's upload step to report the version in the workflow configuration and use the VERSION variable for the artifact name as well as for finding the artifact itself.

 

build-app.ps1; Build script:

$version = Get-Content ./version.txt -Raw 
$version = $version -replace "`n|`r",""
$file = "app-" + $version + ".exe"
echo $version | Out-File $file

echo "::set-env name=VERSION::$version"

Workflow:

name: Main
on: [push]
jobs:
  windows-msvc:
    name: Windows with VC++
    runs-on: windows-latest
    steps:
    - uses: actions/checkout@master
    - name: build
      run: ./build-app.ps1
    - name: upload app version ${{ env.VERSION }}
      uses: actions/upload-artifact@master
      with:
        name: app version ${{ env.VERSION }}
        path: app-${{ env.VERSION }}.exe

Output:

2020-01-13T04:38:44.7160711Z ##[group]Run actions/upload-artifact@master
2020-01-13T04:38:44.7160906Z with:
2020-01-13T04:38:44.7161000Z   name: app version 1.3.45
2020-01-13T04:38:44.7161108Z   path: app-1.3.45.exe
2020-01-13T04:38:44.7161195Z env:
2020-01-13T04:38:44.7161283Z   VERSION: 1.3.45
2020-01-13T04:38:44.7161369Z ##[endgroup]
2020-01-13T04:38:45.0109618Z Uploading artifact 'app version 1.3.45' from 'd:\a\read-artifact-version-from-file\read-artifact-version-from-file\app-1.3.45.exe' for run #13
2020-01-13T04:38:45.0978278Z Uploading 1 files
2020-01-13T04:38:45.4376132Z File upload complete.
2020-01-13T04:38:45.4393771Z Uploaded '8' bytes from 'd:\a\read-artifact-version-from-file\read-artifact-version-from-file\app-1.3.45.exe' to server
2020-01-13T04:38:45.5452933Z Associated artifact app version 1.3.45 (692459) with run #13

 

@lelegard, looks like @brunchboy's input will be the solution along with reading the version from your header file.  With that you can set the environment variable for future steps.  Pretty cool indeed.

 

Please let us know how it turns out!!

Highlighted
Pilot Lvl 1
Message 8 of 9

Re: GitHub Actions: Using non-fixed names in upload-artifact

Pretty cool indeed. It works.

The piece I was missing was the echo "::set-env name=..." trick. 

Thank you all !

Highlighted
Pilot Lvl 1
Message 9 of 9

Re: GitHub Actions: Using non-fixed names in upload-artifact

For reference, if anyone is interested in seeing the result on a real project, the corresponding YAML file is the following: https://github.com/tsduck/tsduck/blob/master/.github/workflows/nightly-build.yml

 

I finally kept a constant name for the artifact name but use the version number in the artifact file name. See this excerpt from the above referenced file:

 

    - name: Build installers
      run: |
        build/Build-Installer.ps1 -NoPause -NoLowPriority -NoSource
        $version = (Get-ChildItem -Recurse -Path installers -Include "TSDuck-Win64-*.exe" | `
          ForEach-Object { $_.Name -replace 'TSDuck-Win64-','' -replace '.exe','' } | `
          Select-Object -Last 1)
        Write-Output "::set-env name=VERSION::$version"
    - name: Upload 64-bit installer
      uses: actions/upload-artifact@master
      with:
        name: installer-64
        path: installers/TSDuck-Win64-${{ env.VERSION }}.exe
    - name: Upload 32-bit installer
      uses: actions/upload-artifact@master
      with:
        name: installer-32
        path: installers/TSDuck-Win32-${{ env.VERSION }}.exe

 

Related topics: