GitHub package registry as Maven repo with many private libraries

I want to use the GitHub package registry as a Maven repository for a multi-git-repo private project. Say I have app1, app2, lib1, lib2, both apps require both apps each.

Currently, I have to declare a maven repositoy for each of the libs I’m using in the apps, e.g. like this:

build.gradle:

repositories {
    mavenCentral()
    maven {
        name = "GitHubPackages-lib1"
        url = "https://maven.pkg.github.com/OWNER/lib1"
        credentials {
            username = System.getenv("GITHUB_USERNAME")
            password = System.getenv("GITHUB_TOKEN")
        }
    }
    maven {
        name = "GitHubPackages-lib2"
        url = "https://maven.pkg.github.com/OWNER/lib2"
        credentials {
            username = System.getenv("GITHUB_USERNAME")
            password = System.getenv("GITHUB_TOKEN")
        }
    }
}

One can easily see that this will not scale well. 

With NPM packages, I get a much better developer experience. There, I can provide something like 

@OWNER:registry=https://npm.pkg.github.com

in my application’s .npmrc and will have access to all published NPM packages in all repositories of OWNER, where that user token has access to. Very convenient!

It would be really nice to have something similar for Maven packages as well.

I have read that I can publish multiple packages to one github repository, so maybe I could setup my own OWNER/mavenrepo and publish all libraries into that. But then the published packages would not be visible on the packages tab of the library, right?

10 Likes

I definitely support the idea of having one central maven repository for all projects. Would really be great.

Publishing to one repository from different other repositories (projects) didn’t work for me. I received a HTTP error 422 trying to do so.

2 Likes

Hi,

You should be able to publish to another repository. 

However it has be in a different package name.

Package names have to be unique across repositories in an organization

You need another token to publish GITHUB_TOKEN will not work. So I had to set up another secret

I just came across this behavior, and I must agree that it is pretty awful. Maven has the scm tag (see docs here), so that could be used by Github in the same way the publishConfig is used in the land of npm. This can also be done via gradle.

Even without introspecting on the scm tag, Java packages are inherently namespaced (unlike Node packages), so publishing at least at an org / user level should be easily doable without worrying about namespace conflicts.

The current implementation is a pretty poor one for anyone using Java.

2 Likes

We’re planning to fix this by removing the required repo scope and following guidance like you’ve suggested. That should make sharing packages much easier.

10 Likes

Great! Do you have an ETA for that?

3 Likes

I’m trying to figure out how to do this as well. The closest I could get to documentation wise was this: https://help.github.com/en/packages/using-github-packages-with-your-projects-ecosystem/configuring-apache-maven-for-use-with-github-packages#publishing-a-package

Either way, how do I do this in a Gradle Groovy script?

I believe the current Maven implementation only takes into account the repository name for SNAPSHOT versions. If you’re using non-SNAPSHOT versions, you could probably get away with the following:

Note, * isn’t special, it could equaly well be ANYTHING. This is likely to stop working if/when using simply https://maven.pkg.github.com/OWNER is supported.

3 Likes

I just ran into the same HTTP 422 error when trying to create a “central” Maven repository in my organization. I want to publish packages from different source repositories into a single Maven repository. I am using Gradle for these tasks. It is rather annoying to manage multiple Maven repositories when using multiple packages. Also, for CI I have to invite a technical user to all the repositories.

2 Likes

@jcansdale could we get an update on this? Is support for https://maven.pkg.github.com/OWNER still being worked on?

4 Likes

Came here searching about this issue and I am also interested in this solution.

@jcansdale Any updates here? Feels a little odd to me to add like 20 repositories just to mimic a global repository for all my dependencies.

2 Likes

just ran into this. getting the ambiguous 422 error. @clarkbw @jcansdale any updates …? anybody?

I solved this by creating a dummy repository called artifacts, then I use a personal access token created for my user, that has access to all repos and permission to write packages. I use that PAT and my user in maven settings and gradle properties to be able to publish to this central dummy repository.

Here is how my Maven settings looks like:

<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 https://maven.apache.org/xsd/settings-1.0.0.xsd">
    <servers>
        <server>
            <id>github</id>
            <username>${github_user}</username>
            <password>${github_personal_token}</password>
        </server>
    </servers>
    <profiles>
        <profile>
            <id>me</id>
            <repositories>
                <repository>
                    <id>github</id>
                    <name>GitHub Packages</name>
                    <url>https://maven.pkg.github.com/${organization}/artifacts</url>
                </repository>
            </repositories>
        </profile>
    </profiles>
</settings>

You can then inject these in your GitHub actions from an org secret, like the following:

      - name: Deploy
        env:
          MAVEN_SETTINGS: ${{ secrets.MAVEN_PUBLISHER_SETTINGS_XML }}
        if: github.ref == 'refs/heads/main'
        run: |
          echo $MAVEN_SETTINGS > settings.xml
          ./mvnw --batch-mode --settings settings.xml deploy

It’s not the ideal solution and I’m looking forward to being able to not have to provide the repository name in my Maven and Gradle configurations. But it is a workaround that works.

1 Like

How do you manage repo purging?!

Means: Remove all but latest SNAPSHOP and maybe only keep the latest 3 released versions of any lib on maven repo?

I’m still searching for a matching github actions script that I can run once a day via cron/scheduled action…

The team investigated dropping the repo section requirement. Unfortunately this would made made other elements in the path ambiguous.

The repo section is only significant when publishing package. When downloading packages, the repo section is ignored:

This means you can do this:

  url = "https://maven.pkg.github.com/OWNER/*"

(The * is just a placeholder and could be anything)

I didn’t run into this problem yet, but I would use the GitHub API for Packages and write a Node script for it. You can manage all of it, listing, deleting, etc.

I am not sure how relevant this is, but we found out that we do not need to specify the repository of every single package, but just one random repository per organisation. So we do publish the packages to the belonging repostory, but we use our meta-repositories orga/.github to add as a maven repositories. That way we just need one repository per organisation (1-2) and not per lib (10+)

As we contacted github they said that indeed this was not intended, but they liked the way this was used and so would keep this ‘feature’ available. They also said that they would add documentation about this, but I do not know if that already happened yet :slight_smile:

Maybe this helps for you?

Can you provide an example configuration for the meta-repository part?

I guess any repository will do. What we found out is that from any repository in the organisation you can retrieve all the packages in this organisation. Maybe just try it out by removing all but one link and see if you can still get all libs? We just used the meta-repo as it is more readable than if you just have any random github-repository in your repositories list.

We just added a normal internal repo called ‘.github’ for clarity.

Does that make sense?