GitHub Actions: Using non-fixed names in upload-artifact #26959
-
Hi, How could I upload an artifact the name of which is not known when writing the YAML file? More precisely, the typical use case is a CI which builds the binary package of an application. The name of the package file contains the version number of the application. This version number has been found somewhere or have been computed somehow by the build procedure. At the end of the build procedure, we know this version number and we know the package file name. But how can we specify the file name to upload-artifact? When hard-coding the names in the YAML file, this would be something like:
But, of course, this is not appropriate because, next time, the version number will be 1.3.46 or 1.4.0 or 2.0, who knows? It could be something like:
But, where do we set the environment variable VERSION? If set in a previous step, the environment is not propagated to the next step. I do not see any possibility to run code in the same step as the upload (“run” and “uses” cannot be used in the same step). I hope you get the idea. Any solution would be appreciated. Thanks -Thierry |
Beta Was this translation helpful? Give feedback.
Replies: 11 comments
-
Hey, @lelegard! Good question. If the step creating the version and the step needed the version are both run in the same job, you can set the environment variable at the job level. This worked for me:
This way the build and upload steps both get the same values for env.VERSION. Your script will read it from the environment. To test I used this as
If the version info is needed in a different job, you will need to create an artifact for the value and upload it in the first job, then download it and use it in the next job. Hope this helps! Let us know how you fare. |
Beta Was this translation helpful? Give feedback.
-
Hi @managedkaos, Thank you for you answer. In my case, specifying the version number in an “env:” section of the YAML file is not an option. The YAML file is not supposed to change. I want automation, not manual modification of a file any time something has changed. Typically, the version number is in a header file (.h, this is a C++ project). The minor version is automatically incremented through git hooks each time a commit is processed. The build script can read the version number but I do not want any manual operation. So, in the job, one step builds the application and “discovers” the version number. The artifact file contains the version number in its name (eg. app-1.3.45.exe). In a later step in the same job, I want to upload this artifact. So, the version number must come from a previous step in the job but it shall not need any manual modification of the YAML file. In that case, the artifact is meant to be publicly downloaded by users. It is not used by another job. This is why keeping the version number in the artifact name is important for the user. |
Beta Was this translation helpful? Give feedback.
-
I think the piece you are missing is that steps in a job can set environment variables for use by later steps in the job by echoing specially-formatted strings, as documented here. I do that all over the place in my own workflow, for reasons just like you are describing. The YAML file doesn’t need to hard-code all environment variables, earlier steps can set them up based on whatever process you need. |
Beta Was this translation helpful? Give feedback.
-
Agree! The workflow should take care of the settings automatically and not require manual intervention. If the version data is in a file, I think the key will be reading it from a file and using it in your build steps vs using an environment variable. I see one suggestiont that mentions how a step can update an environment variable that can be used in a future step. However, that won’t work in the case of using the artifact upload action since the variable would need to be in the configuration at the Workflow level. Instead, you’ll likely need to do two things:
Here’s an example I got working: version.txt; Version file (my stand in for your header file):
build-app.ps1; Build script:
Workflow:
Output:
On another note, you may also consider creating a release along with uploading an artifact. Artifacts are only made avaialable for 90 days and the amount of storage avialable for artifacts is limited by the type of GitHub account you have. Also, you might be able to share informationf or downloading releases more programatically vs sharing and/or using links to download artifacts as there’e no API at the moment to support that; you and your uses will have to go directly to the run that created the artifact and download it from there or you’ll have to manually track down the link to the artifact and share that. Please keep the discussion going; let us know how things work out. -MK |
Beta Was this translation helpful? Give feedback.
-
Actually, @managedkaos using the mechanism I mentioned whereby a step can set an environment variable for use in subsequent steps and jobs does work even when you are using actions like the artifact upload action, because you can use the workflow expression syntax to interpolate the environment variables into the action configuration, and the values set by your previous step will, in fact, be used. I do exactly that in my own workflow. Here is where I set some environment variables in an earlier step, and here is where I use those values in a later step with an upload action. It’s really an amazingly powerful and useful framework GitHub has set up for us here! :slight_smile: |
Beta Was this translation helpful? Give feedback.
-
Yep, so it is @brunchboy! That’s pretty cool. 😃 I’ve validated by making the following changes:
build-app.ps1; Build script:
Workflow:
Output:
@lelegard, looks like @brunchboy’s input will be the solution along with reading the version from your header file. With that you can set the environment variable for future steps. Pretty cool indeed. Please let us know how it turns out!! |
Beta Was this translation helpful? Give feedback.
-
Pretty cool indeed. It works. The piece I was missing was the echo “::set-env name=…” trick. Thank you all ! |
Beta Was this translation helpful? Give feedback.
-
For reference, if anyone is interested in seeing the result on a real project, the corresponding YAML file is the following: https://github.com/tsduck/tsduck/blob/master/.github/workflows/nightly-build.yml I finally kept a constant name for the artifact name but use the version number in the artifact file name. See this excerpt from the above referenced file:
Related topics: |
Beta Was this translation helpful? Give feedback.
-
Thanks for the pointer, @managedkaos! One single build can produce multiple artifacts. For instance, I can build a 32 and a 64-bit electron-based app on Linux with one command. What would be the preferred way to handle and upload more than one artifact with a non-fixed name. I am migrating an Azure pipeline to GH Actions; on Azure, I could upload a dir, and it kept all the artifact names. Something like this.
Thank you! |
Beta Was this translation helpful? Give feedback.
-
it might sound counterintuitive, @kittaakos, but can you give more details on how the artifacts with non-fixed names are named? In your example, it looks like the names follow some pattern like:
If you need to upload each one of these artifacts individually, it seems like your build step could be passed the varaibles as needed for the upload. if you need to upload the artifacts as a bundle of zips as shown in your example, you could create the zips and then create an artifact that is a zip of the zips. Hope that helps to inspire a solution! |
Beta Was this translation helpful? Give feedback.
-
Thanks for your time, @managedkaos. I’ve opened a follow-up (and have closed it already) here: actions/upload-artifact#80 Long story short: it turned out, my requirement is rather a lack of UI feature. |
Beta Was this translation helpful? Give feedback.
Yep, so it is @brunchboy! That’s pretty cool. 😃
I’ve validated by making the following changes:
build-app.ps1; Build script:
Workflow: