docs site: serve a robots.txt at the site root

This commit is contained in:
inference-bot committed 2026-11-03 19:31:12 +00:00
1 parent 44f040c27f
commit fa1f38ced2
2 files changed
+6

No files matched your search

+3
View File
@@ -0,0 +1,3 @@
# Documentation site crawler rules
User-agent: *
Allow: /