Store proprietary binaries in private repo, and use in workflow with actions/checkout

I have a problem, and an idea of a solution, that I’d love some feedback on.

My workflow uses proprietary binaries unavailable in any NuGet repositories, and I cannot myself distribute them legally. Therefore, I cannot build my application in a public repo using GitHub Actions, because I would have to somehow upload the binaries to the build environment, making them publicly available in the process. I could make a private package and keep it in the GitHub Package Repository for my repo, but I need a simpler, and free, alternative.

What if I stored the binaries in a different private repository, and whenever my workflow is triggered the repo downloads the binaries using actions/checkout (with a private GitHub token). This way I don’t distribute the binaries, but can still use them in my workflow.

Are there any obvious drawbacks here? Recommendations against this approach? People who clone my repo obviously will have to be aware that they need to themselves provide the binaries if they are to build the application.

So, I just noticed that Package registry has a free tier for up to 500MB. That should work.