Partial support for maven SNAPSHOTs

Hello,

This answer says that it’s possible to upload/push snapshot packages with the same version (e.g., 0.1-SNAPSHOT).

However, it also says that there was a bug that prevents to push artifacts with -javadoc and -sources extension, somehow. Is that still an issue?

Because I’m trying to upload/push version 3.0.1-SNAPSHOT, and I get the following error:

* What went wrong:
Execution failed for task ':compiler:publishMavenJavaPublicationToMavenRepository'.
> Failed to publish publication 'mavenJava' to repository 'maven'
   > Could not GET 'https://maven.pkg.github.com/neow3j/neow3j/io/neow3j/compiler/3.0.1-SNAPSHOT/maven-metadata.xml'. Received status code 400 from server: Bad Request

When I go to fetch the maven-metadata.xml file on the URL from above, I get:

error retrieving metadata for snapshot file: compiler-3.0.1-20200527.103236-1-sources.jar

What’s the problem?

Thanks!

1 Like

Hi @gsmachado,

Sorry about the delay in getting back to you!

This issue should now be fixed.

I am able to fetch your maven-metadata.xml using the following command:

$ curl https://maven.pkg.github.com/neow3j/neow3j/io/neow3j/compiler/3.0.1-SNAPSHOT/maven-metadata.xml -u token:${READ_PACKAGES_TOKEN} | xmllint --format -
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1661  100  1661    0     0   3831      0 --:--:-- --:--:-- --:--:--  3827
<?xml version="1.0"?>
<metadata modelVersion="">
  <groupId>io.neow3j</groupId>
  <artifactId>compiler</artifactId>
  <version>3.0.1-SNAPSHOT</version>
  <versioning>
    <snapshot>
      <timestamp>20200527.103236</timestamp>
      <buildNumber>1</buildNumber>
    </snapshot>
    <lastUpdated>20200527103248</lastUpdated>
    <snapshotVersions>
      <snapshotVersion>
        <extension>jar</extension>
        <value>3.0.1-20200527.103236-1</value>
        <updated>20200527103237</updated>
      </snapshotVersion>
      <snapshotVersion>
        <extension>pom</extension>
        <value>3.0.1-20200527.103236-1</value>
        <updated>20200527103238</updated>
      </snapshotVersion>

READ_PACKAGES_TOKEN is a PAT with the read:packages scope.

Any chance you could point me to a GitHub Actions workflow that shows this failing?

2 Likes

Hello @jcansdale

Thanks for your reply, even if took a while. No worries! :wink:

So… I can confirm that now I can successfully publish 3.0.1-SNAPSHOT – i.e., update/re-upload artifacts using the same version (like an overwrite). That’s fine!

However, I still don’t get it: even if the repo is public (open-source project), users mandatorily need to use a token with read:packages scope, somehow?

I don’t know if you’re familiar with Java/Gradle ecosystem, but, if I set-up a project with the following gradle.build content, it fails:

plugins {
    id 'java'
}

group 'org.example'
version '1.0-SNAPSHOT'

repositories {
    mavenCentral()
    maven {
        url "https://maven.pkg.github.com/neow3j/neow3j/"
    }
}

dependencies {
    compile 'io.neow3j:contract:3.0.1-SNAPSHOT'
    testCompile group: 'junit', name: 'junit', version: '4.12'
}

The output of a simple gradle compileJava on this repository example gives the following output:

> Could not resolve all files for configuration ':compileClasspath'.
   > Could not resolve io.neow3j:contract:3.0.1-SNAPSHOT.
     Required by:
         project :
      > Could not resolve io.neow3j:contract:3.0.1-SNAPSHOT.
         > Unable to load Maven meta-data from https://maven.pkg.github.com/neow3j/neow3j/io/neow3j/contract/3.0.1-SNAPSHOT/maven-metadata.xml.
            > Could not get resource 'https://maven.pkg.github.com/neow3j/neow3j/io/neow3j/contract/3.0.1-SNAPSHOT/maven-metadata.xml'.
               > Could not GET 'https://maven.pkg.github.com/neow3j/neow3j/io/neow3j/contract/3.0.1-SNAPSHOT/maven-metadata.xml'. Received status code 401 from server: Unauthorized

If this is really the case, and I got it right, this is a real problem for the adoption of Maven-related projects! It will certainly bring lots of frustration, and people will not widely use it. :frowning_face:

People (users of libraries) just want to add the repository URL, add the dependency, and then start using libs/artifacts! That’s it. No generation of tokens. No account on GitHub. No addition configuration. Nothing. It’s open-source software anyway, thus, packages and its metadata (maven-metadata.xml) should be widely accessible. :slight_smile:

Let me know if I misinterpreted something… also, maybe there’s a configuration, somewhere, that I missed.

Thanks!

@gsmachado,

From what I understand, it’s currently intended more for internal development dependencies than for hosting public packages. Packages can be made accessible to anyone with a GitHub account who is motivated to create a PAT (which isn’t exactly public).

It is possible to create a read:packages token on behalf of users that might want to build your project. There are however a couple of complications:

  1. GitHub will automatically delete tokens that are pushed to public repositories
  2. The read:packages scope gives access packages associated with private as well as public packages

I’m still experimenting with Gradle configurations, but what you might be able to do is something like the following:

  1. Create a PAT with the read:packages scope from a safe account (e.g. a machine user account)
  2. Use docker run jcansdale/gpr encode <PAT> to encode the token (see here)
  3. Add a credentials element to the maven details with username and password
  4. username can be anything and password the encoded string from example .npmrc file

For example:

repositories {
    maven {
        name = "remote"
        // Adapt the URL for your remote repository
        url = uri("https://maven.pkg.github.com/jcansdale-test/gradle-java-publish")
        credentials { 
            // Use this if the repo requires auth 
            // see https://docs.gradle.org/6.4/userguide/declaring_repositories.html#sec:supported_transport_protocols
            username = "token"
            password = "\u003c\u0050\u0041\u0054\u003e...................................................."
        }
    }
}

I’d be interested to know if something like this would work for you.

Hey @jcansdale

Thanks for the fast reply! Much appreciated! :slight_smile:

Oh, ok. This is exactly what I meant.
So, it means that GitHub packages feature is not intended for hosting (real) public packages. Gotcha.

However, this is a use case that is interesting for tons of open source projects out there, including neow3j. The advantages of publishing to GitHub Packages instead of JFrog/SonaType/etc are obvious: everything in a single platform, GitHub. It’s faster, and there’s a bunch of opportunities to explore there (including metrics of download, etc) in the entire software delivery lifecycle: development, release, distribution, measure.

At the moment, what you propose is not really suitable for widely public packages – because of the necessity of creating a PAT.

Where can I submit a Feature Request for this?

What if you create a read:packages PAT that is distributed with the project? Is the solution proposed in the PR above not an option?

This is a known issue and being worked on for the next version of GitHub Packages. I’m afraid I can’t give you a timeline though.

Unfortunately, that’s not an option… because I would need to make that read:packages PAT public on the README.md of our project – developers/users should want to copy and paste the details to configure their gradle.build to import our library and start using it. So, since you mentioned that GitHub removes/deletes tokens that are publicly distributed, then, I conclude that’s not a viable option.

Great! We really want this feature. Let us know once it comes out.

If there’s any link or GitHub issue that I can subscribe to follow the progress of such a feature, let me know!

Thanks a lot for your help and directions. :smile:

There isn’t I’m afraid. It would be nice it there was a public repository where we could track this kind of thing. What do you think @whitneyimura?

1 Like