Being DRY: centralized workflows

Code to Cloud GitHub Actions
#1

We are currently using Jenkins, and we make use of their Shared Libraries feature where (part of) the pipeline specification lives in a separate private Git repository. This way you do not have to duplicate CI/CD pipelines among a lot of similar projects which have the same build process, and makes it easy to manage these pipelines in a central location. Especially when you have many projects this is a killer feature.

We are investigating Actions as a possible alternative to a self-hosted Jenkins server, so I am wondering how I would set up such a centralized workflow here… but I have not found anything useful yet. Is this currently possible with Actions? It looks like you can share individual actions among workflows by putting them in a separate public repository, but not the workflows themselves. Keeping the actions themselves private would force you to duplicate them as well, as they then have to live in the same repository as well.

This is the only thing that really keeping us from switching, because otherwise it looks like this will save us a lot of trouble maintaining our own CI server. 

32 Likes
External workflow configuration
Default Organisation-wide Github actions?
#2

We have a quite similar requirement. Several repositories use the same workflow structure. Is there any recommendation regarding how to avoid redundancies by having to maintain the same YAML file(s) multiple times? Thanks for your suggestions.

2 Likes
#3

Hey @bitfactory-henno-sch and @guite, 

Thank you for being here! As I know there isn’t a provision to accomplish this as of now, we do understand this optimization makes sense and there is an open feedback issue for the team to review, to which I’ve added your input. 

8 Likes
#4

my use case

I also have the same need.
There are multiple repositories, between 10-100 or more that could use the same build steps.

In addition of being DRY, being able to change the build steps of those 100 repositories at once is a powerful tool when managing the CI of modular applications where each module is in a separate repository. The modules could even be developed by another party and the module should be tested with latest code of our repository run by our CI script when changes are done in the other party’s repository.

similar issues

It seems that the issue here is similar: External workflow configuration

workarounds

It seems that it could be possible to use the method provided here as a workaround: Trigger a workflow from another workflow
In this method the many repositories can trigger a workflow in another repository, potentially with some parameters.
The downside is that every tested repository needs to have (write?) access to the repository that executes the CI. I am unsure if its possible to report back the success or failure to the initiator of the CI test.

Second workaround that I have used previously for unifying CI of multiple branches has been to make a single script that spawns a job every day or every week for every repository that is defined. The script then runs the same steps - with given parameters. The CI script could even run a setup script from each branch, which can be unique to that branch.
The downside is that every tested repository needs to be added to this CI script and that the tests cannot use events, such as pull request, push and so.

Third workaround is to make all steps into bash scripts that are hosted in another repository. These bash scripts are then cloned and run. The same can be done with an action.
The downside is that we cannot configure the entire workflow, only some steps of it.

Fourth workaround is to just push new workflows with automation, such as https://github.com/marketplace/actions/deploy-github-actions
The downside is that pushing requires access to the repository and listing of the repositories to push to.

Feedback

Seems github requests filling a form for feature requests and feedback:

If you have feedback or feature requests for GitHub Actions, share those in the Feedback form for GitHub Actions.

2 Likes
#5

It looks like this is on the roadmap: https://github.com/github/roadmap/issues/98

However, I’m not looking forward to paying more than 5 times I do now (Team subscription) for this feature, please tell me you won’t limit this to Enterprise :confused:

#6

I’m part of an open-source project maintainers team and we currently rely on Travis Shareable configuration pattern to segment our CI configurations. We consider migrating from Travis to GH Actions. We have 70+ repositories. Many repositories reuse the same CI configuration and other with slight differences. This feature allows us to segment the different reusable parts and assemble them in each repository how we see fit.

It would be awesome to have the same feature with the GH Workflows.

#7

Just in case if someone missed this like me, GitHub actions now support “composite steps” Creating a composite run steps action - GitHub Docs

This will not help you to keep the matrix centralized but will reduce the amount of the code which need to be written

1 Like
#8

This is on GitHubs roadmap. You can already do basic composite workflows to avoid duplication.

1 Like
#9

I’ve written a blog post about this specific issue, and the open-source solution we developed to address the issue.

The build.yaml from repo-1 is and should be identical to the same file in repo-2

#10

GitHub blog had the following in this topic this week: