-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
mongodb只生成了Fans和Follows两个表,然后爬数据一直显示302,没有爬到数据。登录又显示成功,cookie获取成功,哪位高手解答下,万分感谢! #65
Comments
需要将spider中所有http改成https即可 |
1、已将spider中所有http改成https即可; 但是获取cookie后弹出系统错误:python.exe 已停止运行 问题签名: 程序并没有报错,如何处理求指教 |
我不是使用的这个 browser = webdriver.PhantomJS 用的火狐的driver |
mongodb只生成了Fans和Follows两个表,然后爬数据一直显示302,没有爬到数据。登录又显示成功,cookie获取成功,哪位高手解答下,万分感谢!
登录提示:
2017-11-07 10:45:58 [Sina_spider1.cookies] WARNING: Get Cookie Success!( Account:我是马赛克 )
2017-11-07 10:45:58 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.sina.com.cn
2017-11-07 10:45:58 [urllib3.connectionpool] DEBUG: https://login.sina.com.cn:443 "POST /sso/login.php?client=ssologin.js(v1.4.18) HTTP/1.1" 200 None
2017-11-07 10:45:58 [Sina_spider1.cookies] WARNING: Get Cookie Success!( Account:我是马赛克 )
爬内容时提示:
2017-11-07 10:46:00 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (302) to <GET https://weibo.cn/5235640836/follow> from <GET http://weibo.cn/5235640836/follow>
2017-11-07 10:46:40 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (302) to <GET https://weibo.cn/5235640836/fans> from <GET http://weibo.cn/5235640836/fans>
2017-11-07 10:46:59 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
The text was updated successfully, but these errors were encountered: