Dependency across multiple private repos

Code to Cloud GitHub Actions
#1

Is it possible to define dependency across multiple private repositories and block workflow in dependent repositories if build is in progress in any of the dependencies?

We have multiple API repositories which are dependent on multiple business logic repositories and would want to block API repository build if any of its dependencies build is in progress.

We are using GHES 3.4.1.

#2

This post was flagged by the community and is temporarily hidden.

#3

There are choices.

Note that if you’re using GitHub hosted builders, then “blocking” will probably result in being billed while you wait.

Sleep (expensive at runtime)

  1. Write an action that can check the state of a commit in a repository (it would need a PAT or an App Token) and if the state is “pending” or something like it do the next step
  2. Sleep (this is a billable thing) and then go to 1 (this is conceptual, you’re in a loop that sleeps until 1 says go ahead)
  3. Run your work as usual

Message passing and queueing

  1. Write an action that can check the state of a commit in a repository (it would need a PAT or an App Token) and if the state is “pending” or something like it do the next step

  2. Use that action, and use toJSON of the context for the thing that you need to stall. Use actions/upload-artifact to publish it w/ a well known name.

  3. Send the reference to the stashed object to the repository that’s pending (you could do this by adding a comment to the commit that’s building with the url for your current job run)

  4. End your workflow – There will be another entrypoint later, but this job is done.

  5. In the workflow for the in progress side, when it’s about ready to be done, have it look for any comments on the commit it’s finishing and then use a workflow_dispatch (using another PAT) to trigger a workflow in that first repository for each of them

  6. your job is done

  7. use the workflow_dispatch event w/ the job from the previous step (which in turn came from step 2) to retrieve and fromJSON the data.

  8. do the job you would normally have done at step 1

Self-hosted

  1. In your dependent repository, update your self-hosted runners once a commit is happy to include the label for the commit

  2. Have a job that defines a self-hosted runner label based on the dependent commit

  3. Have your other jobs defined to use that label

There should be various ways to make this work, but it might take a bit of creativity.

Note that I haven’t done any of this, so these are very much back of the napkin answers.

#4

Thanks for response. I could not understand steps for self-hosted runners and commit label. Please provide more detail.
We have a bunch of self-hosted runners shared across the organization.

#5

Basically each runner has a set of labels, they’re designed to allow you to control which jobs they’ll accept:
https://docs.github.com/en/actions/hosting-your-own-runners/using-labels-with-self-hosted-runners

Here you’d basically ensure that your runners grow a new label when they’re ready to handle something that has passed in the one repository, and then have your jobs rely on that label to stall until it’s available.

#6

Interesting approach. It would require custom runners for our set of repositories.

#7

I’m pretty sure it’s cheaper to implement than any fancy queuing mechanism, since you’re just relying on GitHub’s automatic queuing.

Note that you can just add extra labels to existing runners, you don’t need to start new runners:

https://docs.github.com/en/rest/actions/self-hosted-runners#add-custom-labels-to-a-self-hosted-runner-for-an-organization