Robots.txt is supported?

I’m trying to place a robot’s.txt in root folder but it’s not recognized when I access from any browser. What I can do?

1 Like

Make sure your file is named robots.txt, next git add, commit and push to GitHub. Check if you can see robots.txt in the GitHub repo through your browser. Go to the environment section and see in Activity log when it was last deploy. Maybe something blocks changes.

In my case I use jekyll-sitemap gem (https://github.com/fractalsoft/blog.fractalsoft.org/blob/master/_config.yml#L32) and it generates robots.txt with sitemap (https://github.com/jekyll/jekyll-sitemap/blob/master/lib/robots.txt).

After deploy it looks like this:
https://blog.fractalsoft.org/robots.txt

Let me know if this helped.

1 Like