This is the old Medusa fork of the Anemone gem.
See the new and revamped Medusa Crawler gem instead: https://github.com/brutuscat/medusa-crawler
Medusa is a web spider framework that can spider a domain and collect useful information about the pages it visits. It is versatile, allowing you to write your own specialized spider tasks quickly and easily.