本程序由python语言编写.
1. 依赖库见(dependencies)
2. 使用scrapy爬虫框架,mongo为数据存储介质,redis作为缓存, 利用celery+redis实现消息队列.
3. 启动
A. 修改配置: (settings.py)
B. easy_install celery 或 pip install celery 后进入到项目根目录, 运行celery worker --app=utils.celeryapp:app -l info -f ./log/celery.log & 启动celery.
C. 运行脚本(dispatcher.py) 启动整个程序.
Created with Nodeclipse (Eclipse Marketplace, site)
Nodeclipse is free open-source project that grows with your contributions.