Uploading / Committing / Pushing 10GB+ Repository

Hello,

My team and I have a 10 GB+ project we would like to push to an existing Github Repository. It has many directories, sub-directories, and files. I have tried both visual tools as well as command line tools (gitbash) to get this done but I have not been successful. We have tried many times with different results.

1 attempt result) Once, the project was pushed but had a folder icon with an arrow on it. We were not able to access anything else in the project.

2 attempt result) From the visual tool it would give an error similar to “cannot publish to repository”.

3 attempt result ) From command line, the process would get hung up to a percentage and never complete the process.

We have considered splitting the project and uploading folder by folder, but not only does this seem like overkill, but I am concerned since the project is this large, this opens the door to potential dependency issues when another authorized user tries to download (i.e. project will not build properly if a dependency is missing), so I would like to avoid this if possible.

I am a novice using Github and have been successful in adding, committing and pushing, but I have not experienced these type of issues before.

Any help or input is welcome.

Thanks you Github community,
JC

That is a large code base. Do you have binaries or other files of that type in your repo? Do you have files that are over 100MB in your repos?

Hello Mickey,

Thanks for your reply. Yes to both actually. Is there something that I should be doing differently?

Thanks,
JC

GitHub blocks files larger than 100MB as described, that’s described here: Conditions for large files - GitHub Docs

Committing binaries is generally considered bad practice, a Git repository should hold sources, not compiled code. Of course that doesn’t really apply to things like artwork, or audio/video recordings where there is no “source” as in programming. To avoid putting such files in the repository itself there are tools that let you store some form of reference in Git and the large file on some other storage, the approach supported by GitHub is Git LFS (Large File Storage).

2 Likes