-
Notifications
You must be signed in to change notification settings - Fork 189
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
请问一下在执行scrapy crawl baike后,没有一点报错但直接跳出程序是什么情况 #27
Comments
#20 看这个issue |
下午跑了下发现百度百科加验证了,不好爬取 |
大佬,我出现的是这个,你也是吗?啥数据都没爬下来 It is also the default value. In other words, it is normal to get this warning if you have not defined a value for the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting. This is so for backward compatibility reasons, but it will change in a future version of Scrapy. See the documentation of the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting for information on how to handle this deprecation. |
你这报错了,百度防爬会返回一个验证页面,看response内容 |
baike/settings.py 这个里面DEFAULT_REQUEST_HEADERS 换一个ua试试 |
baike/settings.py 这个里面DEFAULT_REQUEST_HEADERS 换一个ua尝试 |
我也遇到了这个问题,有解决方法了吗? |
结果是这个:(base) D:\bandzip\WEB_KG-master\baike>d:/ProgramData/Anaconda3/python.exe d:/bandzip/WEB_KG-master/baike/spiders/baike.py
感觉好像没有爬取到内容,是我什么地方出错了吗。
The text was updated successfully, but these errors were encountered: