Help
cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Copilot Lvl 3
Message 1 of 4

How to discover artifacts that were created/uploaded by a matrix build of a different job?

Hello,

 

I have a CICD workflow with two jobs: Build and Publish. The former is running on a matrix for different Python versions and Docker images:

 

strategy:
  matrix:
    image:
    - quay.io/pypa/manylinux2010_x86_64
    - quay.io/pypa/manylinux2014_x86_64
    python: [3.7, 3.8]

 

Each run of the Build job generates an artifact (a directory with Python wheels) and uploads it with a unique name like this:

 

- name: upload
  uses: actions/upload-artifact@v1
  with:
    name: wheels-${{ matrix.python }}-${{ matrix.image }}
    path: wheels

 

Then, after all the Build jobs are complete, the Publish job runs once to publish the artifacts to PyPI. Here is where I am stuck now. The Publish job doesn't know the exact names of created artifacts created by the Build job because they were created using the matrix expansion. To my understanding, the Publish job does not have access to the strategy matrix of the Build job to somehow expand the values. Furthermore, to my knowledge, actions/download-artifact does not support wildcards. So, something like this won't work:

 

- name: download
  uses: actions/download-artifact@v1
  with:
    name: wheels*

 

Any idea how can I download all artifacts starting with "wheels" in an elegant way?

 

Thank you.

 

3 Replies
Highlighted
GitHub Partner
Message 2 of 4

Re: How to discover artifacts that were created/uploaded by a matrix build of a different job?

Yes, you are right. The Publish job does not have access to the strategy matrix of the Build job to somehow expand the values. 

Is it possible for you to add the same matrix to publish job? Then you could use the same matrix as the artifact name in download artifacts step. 

For example: 

publish job.png

By the way, according to your matrix with image ( quay.io/pypa/manylinux2010_x86_64 ), the artifacts name is not valid. You need to make some change to avoid containing specific characters in artifact name. 

artifact name.png

 

Highlighted
Copilot Lvl 3
Message 3 of 4

Re: How to discover artifacts that were created/uploaded by a matrix build of a different job?

Hi @Yanjingzhu,

 

Thank you for helping me. Yes, I could copy the strategy matrix to the Publish job. That should work. However, this will result in the execution of the Publish job multiple times, which is a bit wasteful. Ideally, I would like to run the Publish job once and have it somehow discover all the artifacts created by the multiple Build jobs.

 

Also, yes, you are right, the image names are not valid artifact names. To be honest, I did the initial tests only with the python version as part of the artifact name xD

Highlighted
GitHub Partner
Message 4 of 4

Re: How to discover artifacts that were created/uploaded by a matrix build of a different job?

According to your scenario, copy the strategy matrix to the Publish job is not suitable. 

I would encourage you to submit an issue here for supporting wildcard in download artifacts action  .