Add the /help page in robots.txt

The /help page has docs which we don't want to be crawled as we prefer the docs website instead. Related https://gitlab.com/gitlab-org/gitlab-ce/issues/44433

Add the /help page in robots.txt
71f707bf · Achilleas Pipinellis · 5790caa7 · 71f707bf
Unverified Commit 71f707bf authored Mar 22, 2018 by Achilleas Pipinellis
Hide whitespace changes
Inline Side-by-side

Showing with 1 addition and 0 deletions

robots.txt public/robots.txt +1 -0

No files found.
--- a/public/robots.txt
+++ b/public/robots.txt
@@ -20,6 +20,7 @@ Disallow: /projects/new
 Disallow: /groups/new
 Disallow: /groups/*/edit
 Disallow: /users
+Disallow: /help

 # Global snippets
 User-Agent: *