Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

启动worker时执行到**[2020-04-02 12:36:58,850: INFO/MainProcess] mingle: all alone**就不再继续 #204

Open
keithkang1986 opened this issue Apr 2, 2020 · 2 comments

Comments

@keithkang1986
Copy link

在提交Issue之前请先回答下面问题,谢谢!

1.你是怎么操作的?

按步骤到第9步 然后执行celery -A tasks.workers -Q login_queue,user_crawler,fans_followers,search_crawler,home_crawler worker -l info -c 1

2.你期望的结果是什么?
启动worker

3.实际上你得到的结果是什么?
C:\data\爬虫\weibospider-1.7.2\config\conf.py:12: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://m
sg.pyyaml.org/load for full details.
cf = load(cont)

-------------- celery@LAPTOP-QD49LR5G v4.4.2 (cliffs)
--- ***** -----
-- ******* ---- Windows-10-10.0.18362-SP0 2020-04-02 12:36:57

  • *** --- * ---
  • ** ---------- [config]
  • ** ---------- .> app: weibo_task:0x14e2983a0b8
  • ** ---------- .> transport: redis://127.0.0.1:6379/5
  • ** ---------- .> results: disabled://
  • *** --- * --- .> concurrency: 1 (prefork)
    -- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
    --- ***** -----
    -------------- [queues]
    .> fans_followers exchange=fans_followers(direct) key=for_fans_followers
    .> home_crawler exchange=home_crawler(direct) key=home_info
    .> login_queue exchange=login_queue(direct) key=for_login
    .> search_crawler exchange=search_crawler(direct) key=for_search_info
    .> user_crawler exchange=user_crawler(direct) key=for_user_info

[tasks]
. tasks.comment.crawl_comment_by_page
. tasks.comment.crawl_comment_page
. tasks.comment.excute_comment_task
. tasks.home.crawl_ajax_page
. tasks.home.crawl_weibo_datas
. tasks.home.excute_home_task
. tasks.login.excute_login_task
. tasks.login.login_task
. tasks.repost.crawl_repost_by_page
. tasks.repost.crawl_repost_page
. tasks.repost.excute_repost_task
. tasks.search.excute_search_task
. tasks.search.search_keyword
. tasks.user.crawl_follower_fans
. tasks.user.crawl_person_infos
. tasks.user.excute_user_task

[2020-04-02 12:36:57,822: INFO/MainProcess] Connected to redis://127.0.0.1:6379/5
[2020-04-02 12:36:57,830: INFO/MainProcess] mingle: searching for neighbors
[2020-04-02 12:36:58,236: INFO/SpawnPoolWorker-1] child process 13708 calling self.run()
C:\data\爬虫\weibospider-1.7.2\config\conf.py:12: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://m
sg.pyyaml.org/load for full details.
cf = load(cont)
[2020-04-02 12:36:58,850: INFO/MainProcess] mingle: all alone

4.你使用的是哪个版本的WeiboSpider? 你的操作系统是什么?是否有读本项目的[常见问题]
weibospiderv1.7.2;操作系统windows10。已读相关问题,但未解决。

@thekingofcity
Copy link
Member

windows并非项目支持, 尝试降级下celery? 我记得4.x就抛弃windows了

@OneCodeMonkey
Copy link
Member

@keithkang1986 win10 下可以起 虚拟机开发,直接拿windows 的 python 环境跑不行的

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants