forked from nevali/crawl
-
Notifications
You must be signed in to change notification settings - Fork 0
Home
Mo McRoberts edited this page Jan 30, 2017
·
3 revisions
Anansi is a framework for building web crawlers, and a web crawler built using that framework.
It consists of:
- libcrawl, a library for implementing web crawlers.
- crawld, a daemon which uses a relational database-based queue.
-
Processors which can be enabled with
crawld, in order to provide additional functionality. - Some utilities built using
libcrawl
See the description of Anansi on Wikipedia.
Anansi is open source software, distributed according to the terms of the Apache License, 2.0.