Update robots.txt to hide older versions of the docs (#3030)

* Update robots.txt to hide older versions of the docs Signed-off-by: Tynan DeBold <thdebold@gmail.com> * Update robots.txt Signed-off-by: Tynan DeBold <thdebold@gmail.com> --------- Signed-off-by: Tynan DeBold <thdebold@gmail.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
kedro-org · Sep 13, 2023 · 0fd1cac · 0fd1cac
1 parent b87af34
commit 0fd1cac
Showing 1 changed file with 13 additions and 4 deletions.
diff --git a/docs/source/robots.txt b/docs/source/robots.txt
@@ -1,5 +1,14 @@
 User-agent: *
-Disallow: *
-Allow: /en/stable
-Allow: /en/latest
-Allow: /en/0.18.*
+Disallow: /
+Allow: /en/stable/
+Allow: /en/latest/
+Allow: /en/0.18.5/
+Allow: /en/0.18.6/
+Allow: /en/0.18.7/
+Allow: /en/0.18.8/
+Allow: /en/0.18.9/
+Allow: /en/0.18.10/
+Allow: /en/0.18.11/
+Allow: /en/0.18.12/
+Allow: /en/0.18.13/
+Allow: /en/0.17.7/