Port sitemap work

[mbarnett]

Caching currently lasts 24 hours from the first request to /sitemap.xml, although we can raise or lower this as desired. My back-of-the envelope math says it will take around 45 seconds to respond to the first request assuming about 45k items in the repository. A nightly cron task to curl the URL would eliminate any lag for most users.

• Get the modified date indexed into Solr
• Theses.
• Item visibility (filter out private items)
• Add a static robots.txt file?
• Other requirements from Google Scholar requirements: Google Scholar optimization #234
• Change SitemapPolicy to simply inherit from ApplicationPolicy.
• Add an application specific property on fileset to include hash, file size, and mimetype information.