Add a User-Agent

And remove / from the Disallow rule

Author: jsha

static/robots.txt

@@ -2,5 +2,6 @@ Sitemap: https://docs.rs/sitemap.xml
 # Semver-based URL are always redirects, and sometimes
 # confuse Google's duplicate detection, so we block crawling them.
 # https://docs.rs/about/redirections
-Disallow: */^
-Disallow: */~
+User-Agent: *
+Disallow: *^
+Disallow: *~