Specific branch for download button

Hi, I have set up a GitHub page with one of the default themes. I added show_downloads: true to _config.yml. When I now click on one of the download buttons on the site and open the contents, I see that it downloaded the gh-pages branch instead of the master branch where the source code lives.

How can I tell GitHub pages to download from the master branch release instead of downloading the gh-pages branch?