I would like to start a workflow when all artifacts from a previous workflow have been published and are publicly available. This looks simple but I have not found any proper solution so far.
The first workflow is a scheduled one, the nightly builds of a project. The second workflow sends a trigger to a server which pulls the artifacts of the first workflow and publish them on its Web site. Simple request: rebuild a project every night and publish the binaries.
I tried several approaches without success. The current setup is very conservative: schedule the “publish” workflow three hours after the “build” workflow, hoping that the “build” phase would complete in the meantime.
It worked so far until the last few days where the “build” workflow was started up to seven hours behind schedule and the “publish” one only four hours behind schedule. The “publish” workflow being actually launched before the “build” one, nothing is published of course. This happened two days in a row.
- Nightly build workflow (3 jobs): 01:10 AM
- Publish artifacts: 04:10 AM
Sample start times on May 31st 2021:
- Nightly build workflow:
- Windows build job: 08:25 AM
- Linux build job: 06:35 AM
- Documentation build job: 06:28 AM
- Publish artifacts: 07:59
I have no objection to workflow being started late, or even very late. Resources are limited, workflows are queued and the service is free anyway. But I do need a deterministic way to serialize the “build” and “publish” phases.
Initially, I wanted to trigger the publication in the same workflow as the build, after the
upload-artifact actions, as a last job, depending on all others. But the artifact is actually published after the completion of the workflow only (reported by several users here). So, the trigger would occur before the publication of the artifact.
Then, I looked for a way to explicitly synchronize or serialize workflows and found none.
Finally, I used the conservative method which consists in scheduling the two workflows independently with a sufficiently long delay between the two. But this method is not deterministic and no longer works due to unexpected huge delays in the workflow scheduling.
So, how would you start a job after the actual publication of the artifacts of a workflow, in a deterministic way? This could be done in the same workflow or in two different workflows, I do not really care. I just need to safely download artifacts when they are ready, from an public server.