scott/crabfit
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Robots and sitemap

Browse source
This commit is contained in:
Ben Grant 2021-04-20 19:32:31 +10:00
parent 96c723a94c
commit c680c22bb3
2 changed files with 17 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

6
crabfit-frontend/public/robots.txt
View file

@ -1,3 +1,7 @@
# https://www.robotstxt.org/robotstxt.html
User-agent: *
Disallow:
Allow: /$
Allow: /how-to$
Disallow: *
sitemap: https://crab.fit/sitemap.xml