GitHub - hkyouwen/image-Crawler: a python2 tool to crawl image

Branches Tags

Name		Name	Last commit message	Last commit date
Latest commit History 1 Commit
Readme.txt		Readme.txt
clist.py		clist.py
crawl.py		crawl.py
seed.txt		seed.txt
spider.conf		spider.conf

Repository files navigation

excute command:
  python crawl.py -c spider.conf


spider.conf说明：
url_list_file:          #输入url列表文件
output_directory：      #输出路径
max_depth               #最大抓取深度
crawl_interval          #抓取间隔
crawl_timeout           #抓取超时
target_url              #目标网页
thread_count            #抓取线程数