-
Currently moving from an on-premise pipeline composed of Bitbucket, TeamCity and Octopus Deploy to Github and loving having a single platform where I can define everything as code - so far so good. But to my surprise and coming from Octopus Deploy, variable management is really a big pain point and pretty much a deal breaker for many teams. And I’m a little puzzled as why this is not discussed more in the community (I may have missed it), making me thing we might have missed something. The main features I’m missing from Octopus Deploy
What we are currently doing in our CICD pipeline in GH is explicitely declaring environment variables for each and every environment (job), using repo and org. secrets for the bits we do not want to expose. This gets even worse when you have adhoc workflows for registrating e.g. idenity metadata for API’s and clients, therefor duplicating the exact same variables into additional workflows. Questions:
Any information is greatly appreciated! |
Beta Was this translation helpful? Give feedback.
Replies: 12 comments
-
Those features, at least when it comes to managing |
Beta Was this translation helpful? Give feedback.
-
@spol-vt could you provide a link with reading material in the docs to that extended variable management capabilities? @jnus now that composed actions have much more flexibility (announced a few weeks back) - I am wondering if you would maybe want to create a sort of repository of configuration where all the data lives in JS files, and use an action that works based on that. Kind of like in the concept of GitOps. You’ll manage those variables simply by editing those JSON files. |
Beta Was this translation helpful? Give feedback.
-
@lirantal Environments - GitHub Docs
Sharing workflows with your organization - GitHub Docs
There is nothing for classic clear txt vars, just secrets, however since it is basically same difference, you can use it for that purpose. With Enterprise Could one can create Org level, Repo level and then Environment level secrets + there is permission management around access to secrets, etc. Functionality is not as extensive as in products mentioned by @jnus, but there is something that goes towards it. |
Beta Was this translation helpful? Give feedback.
-
The secrets per environment are nice if you an enterprise plan. The thing I miss the most is that you can’t have variables that are readable… Why only have secrets? 😅 |
Beta Was this translation helpful? Give feedback.
-
@spol-vt - So some of the feature set is partly there, in terms of secrets, but not on an organization level though where you can’t scope to environments. We share a ton of variables today via Octopus Deploy, scoped for the various environments, ready to use. This could be resolved in others ways than inheritage (submoduling, npm/nupkg etc), so I’m not seeing this as the biggest challange. Not having a unified and context specific ‘view’ of you variables in you workflow(s), in the really big pain point, where you have to declare all environment specific varibles per environment prior to deploying. @lirantal Having all variables as secrets will not be easy to maintain, plus I really applaud having everything in vcs. @ThomasReyskens - It’s a good question - why only have the options to scope secrets per environment and not clear text? My guess is that the GH team also loves having everything in vsc, including configuration and giving the option to save clear text variables on the GH controlplane, violates this principal. A variable framework should definitely based on having the configuration in vsc and having secrets outside, is just a necessary evil to keep sensitive information safe. |
Beta Was this translation helpful? Give feedback.
-
@lirantal - regarding composed actions, that might be an idea going forward. Depending on what kind of initiative is going on around the community, I might start to build a variable framework based on the ideas of Octopus Deploy - e.g. variables in a json structure, scopes to environment/workflow/jobs/steps, supports variable substitution and context aware depending on the environment you are deploying to, workflow etc. But I really want to make sure this is not on the backlog for GH, as I want to use vanilla GH if possible 😑 |
Beta Was this translation helpful? Give feedback.
-
Makes sense, yeah :slight_smile: |
Beta Was this translation helpful? Give feedback.
-
I ended up building a “pre-processing” tool for GHA workflows tailored to my company. This included variables, an equivalent to YAML anchors and pre-populating jobs with common steps. Downside is we have to remember to re-run this process every time we change the input workflow, but it makes it much simpler to manage. |
Beta Was this translation helpful? Give feedback.
-
@dorner interesting way to go about it. If you are allowed to share any details about your current implementation (publicly/privately), that would be greatly appreciated for inspirational purposes. Still really puzzled to why this is not on the backlog for the GH team, with all the focus Microsoft is giving GitHub, as this is a big big pain point with larger deployment setups. |
Beta Was this translation helpful? Give feedback.
-
@jnus you can see a cleaned-up version of it here: https://github.com/dorner/gha-config This version probably doesn’t work per se because I scrubbed some of it, but that’s the general approach. |
Beta Was this translation helpful? Give feedback.
-
So I’ve been working on an action, json-variables, for this particular GH issue, and while its still work in progress before I publish it, I thought I might share it here and maybe get a little input.
GitHub - jnus/json-variablesContribute to jnus/json-variables development by creating an account on GitHub. Let me know what you think. |
Beta Was this translation helpful? Give feedback.
-
Fyi - v1.0 of the action just released → json-variables · Actions · GitHub Marketplace · GitHub |
Beta Was this translation helpful? Give feedback.
So I’ve been working on an action, json-variables, for this particular GH issue, and while its still work in progress before I publish it, I thought I might share it here and maybe get a little input.
GitHub - jnus/json-variables
Contribute to jnus/json-variables development by creating an account on GitHub.
Let me know what you think.