The API doc page can be crawled

RNAcentral · Dec 16, 2024 · 318ea64 · 318ea64
1 parent af30fce
commit 318ea64
Showing 1 changed file with 2 additions and 1 deletion.
diff --git a/rnacentral/templates/robots.txt b/rnacentral/templates/robots.txt
@@ -12,7 +12,8 @@ Disallow: /
 
 User-agent: *
 Crawl-delay: 5
-Disallow: /api/
+Disallow: /api/current/
+Disallow: /api/v1/
 Disallow: /export/
 Disallow: /sequence-search/