Add "rel=noindex" to the redirect page

[cmcfadden]

Right now, we vend a 302 redirect but also output HTML for browsers that don't follow the header, or in cases where the header gets stripped. We should add rel=noindex into that HTML to prevent google from indexing it.

[cmcfadden]

doesn't look like there's a way to override this directly, would probably need to write a new redirect_to function

[cmcfadden]

maybe better to look at a robots.txt for the site?

[jxjj]

Related to #177 ?

robots.txt doesn't seem to be doing anything right now. What's the desired disallow? Everything except homepage? Something like:

# robots.txt
User-agent: *
Disallow: /
Allow: /$