[BUG] NuGet: Support build metadata properly

The repo for the following package is https://github.com/verybadcat/CSharpMath.

After creating a NuGet.Config file as specified (feed is https://nuget.pkg.github.com/verybadcat/index.json), the command dotnet add GitHubReport package CSharpMath.SkiaSharp --version 0.4.2-ci-9db8a6dec29202804764fab9d6f7f19e43c3c083\+2020.06.10-07.22.37 fails:

  Determining projects to restore...
  Writing C:\Users\hadri\AppData\Local\Temp\tmpB503.tmp
info : Adding PackageReference for package 'CSharpMath.SkiaSharp' into project 'C:\Users\hadri\source\repos\GitHubReport_Nightly\GitHubReport\GitHubReport.csproj'.
info : Restoring packages for C:\Users\hadri\source\repos\GitHubReport_Nightly\GitHubReport\GitHubReport.csproj...
info :   CACHE https://api.nuget.org/v3-flatcontainer/csharpmath.skiasharp/index.json
info :   CACHE https://nuget.pkg.github.com/verybadcat/download/csharpmath.skiasharp/index.json
info :   GET https://nuget.pkg.github.com/verybadcat/download/csharpmath.skiasharp/0.4.2-ci-9db8a6dec29202804764fab9d6f7f19e43c3c083/csharpmath.skiasharp.0.4.2-ci-9db8a6dec29202804764fab9d6f7f19e43c3c083.nupkg
info :   NotFound https://nuget.pkg.github.com/verybadcat/download/csharpmath.skiasharp/0.4.2-ci-9db8a6dec29202804764fab9d6f7f19e43c3c083/csharpmath.skiasharp.0.4.2-ci-9db8a6dec29202804764fab9d6f7f19e43c3c083.nupkg 1315ms
info :   GET https://api.nuget.org/v3-flatcontainer/csharpmath.skiasharp/index.json
info :   GET https://nuget.pkg.github.com/verybadcat/download/csharpmath.skiasharp/index.json
info :   OK https://nuget.pkg.github.com/verybadcat/download/csharpmath.skiasharp/index.json 592ms
info :   CACHE https://nuget.pkg.github.com/verybadcat/download/csharpmath.skiasharp/index.json
error: The feed 'CSharpMathNightly [https://nuget.pkg.github.com/verybadcat/index.json]' lists package 'CSharpMath.SkiaSharp.0.4.2-ci-9db8a6dec29202804764fab9d6f7f19e43c3c083' but multiple attempts to download the nupkg have failed. The feed is either invalid or required packages were removed while the current operation was in progress. Verify the package exists on the feed and try again.
error:   Unable to find package 'CSharpMath.SkiaSharp.0.4.2-ci-9db8a6dec29202804764fab9d6f7f19e43c3c083'.

The important line is

info :   NotFound https://nuget.pkg.github.com/verybadcat/download/csharpmath.skiasharp/0.4.2-ci-9db8a6dec29202804764fab9d6f7f19e43c3c083/csharpmath.skiasharp.0.4.2-ci-9db8a6dec29202804764fab9d6f7f19e43c3c083.nupkg 1315ms

Opening the URL

https://nuget.pkg.github.com/verybadcat/download/csharpmath.skiasharp/0.4.2-ci-9db8a6dec29202804764fab9d6f7f19e43c3c083/csharpmath.skiasharp.0.4.2-ci-9db8a6dec29202804764fab9d6f7f19e43c3c083.nupkg

, this line is outputted:

{"error":"Version \"0.4.2-ci-9db8a6dec29202804764fab9d6f7f19e43c3c083\" does not exist on nuget package \"csharpmath.skiasharp\" under owner \"verybadcat\""}

This is because

https://nuget.pkg.github.com/verybadcat/download/csharpmath.skiasharp/0.4.2-ci-9db8a6dec29202804764fab9d6f7f19e43c3c083+2020.06.10-07.23.29/csharpmath.skiasharp.0.4.2-ci-9db8a6dec29202804764fab9d6f7f19e43c3c083+2020.06.10-07.23.29.nupkg

is the actual URL.

Contrast this with e.g. Xamarin.Forms on NuGet.org, version

4.7.0.848-pre3+sha.9bae2b34e-azdo.3764102

, the download URL is

https://api.nuget.org/v3-flatcontainer/xamarin.forms/4.7.0.848-pre3/xamarin.forms.4.7.0.848-pre3.nupkg

, which has NO build metadata in the name.

Therefore, the NuGet GitHub Package registry appending the build metadata to the download URL is incompatible with existing .NET tooling and should be fixed. Thanks.

1 Like

Hi @happypig375,

I didn’t know NuGet supported SemVer2 and metadata!

I haven’t been able to find much documentation about it. Here are a couple of blog entries and the docs was able to find:



https://articles.assembla.com/en/articles/2729137-semver2-support-for-nuget-endpoints-on-myget

I’m interested to know how you’re using metadata and what problem you’re hoping to solve? Do you want to be able to push multiple build of the same version (like Maven’s SNAPSHOT versions) or are you using it to link back to the commit a package version was built from?

Thanks,
Jamie.

3 Likes

Multiple versions only differentiable by build metadata are considered the same version. This is how SemVer works. I am using metadata to store build dates - the package commit is encoded in the prerelease section.

The issue is that the GitHub Package registry is incompatible with existing .NET tooling. It should follow NuGet.org in package URL generation.

@happypig375 am I understanding this correctly.

  • If a NuGet registry supports storing the +metadata as part of a package version (which GitHub Packages apparently does), it should also support returning the package when the metadata itsn’t metadata
  • Alternatively a NuGet registry could simply ignore metadata when storing and fetching a package version

It sounds like the easy fix would be to simply ignore everything after # in a package version. :thinking:

The metadata IS metadata, no matter the content. Just support it properly, thanks.

If NuGet.org does this, then GitHub Packages should too.

The metadata IS metadata, no matter the content. Just support it properly, thanks.

Sorry, I meant when the metadata isn’t specified. :wink:

1 Like

I’m also running into this issue. You can’t even manually specify the build metadata as a workaround either since NuGet ignores it entirely.

I made a quick repository that demonstrates the issue using GitHub Actions:

Basically it publishes two packages to GitHub Packages: One with build metadata and one without. Then in separate jobs it attempts to add those packages to a fresh console project. (With and without explicit build metadata just to show that can’t be used as a workaround.)

Here’s an example run showing the failed restores.

There’s also an additional workflow that demonstrates everything works as expected when NuGet.org is used instead.

1 Like

Hi @PathogenDavid,

Thanks for putting this together!

I noticed your note about source-url needing to be all lowercase.

I’m interested to know why you think this. An upper-case owner seems work for me, see here:

I’ve sent you a PR. :slightly_smiling_face:

No problem, hope it helps!

This documentation:

Because upper case letters aren’t supported, you must use lowercase letters for the repository owner even if the GitHub user or organization name contains uppercase letters.

I do remember trying ignoring that documentation and using github.repository_owner anyway, but it didn’t work at the time. If I had to guess I was unknowingly suffering from the intermittent failures caused by https://github.com/NuGet/Home/issues/9775 and blamed the failure on the URL so I made that little Python script to avoid the hard-coded URL.

Got it merged and looks like it’s working! I also tested it on my real GitHub Actions consumer and it’s working as well.

Thanks for the confirmation! I’ve suggested we update our docs to reflect this. :slightly_smiling_face:

1 Like

The docs have now been updated to not insist on all lowercase! :slightly_smiling_face:

1 Like

@jcansdale The original bug mentioned in the top post has yet to be fixed?

@happypig375,

It hasn’t I’m afraid. :slightly_frowning_face:

A possible workaround would be to use the following when publishing to GitHub Packages:

dotnet tool update gpr -g
gpr push <path to *.nupkg> --version <version without metadata>

That way you could continue to use metadata on nuget.org, but a version stripped on metadata would be published to GitHub Packages.

Thanks for getting the docs issues sorted out!

Unfortunately that workaround didn’t seem to work for me: https://github.com/PathogenPlayground/GitHubPackagesBuildMetadataTest/runs/1004018759?check_suite_focus=true

It appears that gpr is properly re-writing the nuspec to change the version number (this line writes out the right version), but when it actually registers the package the build metadata is back:

[PackageWithBuildMetadata.1.0.2.nupkg]: Repository url: https://github.com/PathogenPlayground/GitHubPackagesBuildMetadataTest. Version: 1.0.2. Size: 3451 bytes. 
...
[PackageWithBuildMetadata.1.0.2.nupkg]: Successfully registered nuget package: PackageWithBuildMetadata (1.0.2+BuildMetadata)

I’ll try to find some time to dig further later, but I thought I’d at least share what I discovered in my quick test.

1 Like

@PathogenDavid,

Thanks for the heads up! Could you open an issue about this to remind me?

It looks like NuGetVersion.ToString doesn’t show metadata by default (which is why the version strings appear different). It would be nice if it did automatically strip any metadata!

Ah, that would certainly do it! Filed an issue here:

I’ve created a PR with a fix for this. :crossed_fingers:

Please see the “How to test” section:

It should automatically strip metadata, so you don’t need to bother with the --version option.

Please let me know how you get on!

1 Like

I replied on the PR, but for the sake of discussion consistency: I can confirm that fixed my issue. Thanks for getting the workaround fixed!

1 Like