Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Healthchecker init error when restart Kong with much worker process #4453

Closed
wuguangkuo opened this issue Mar 29, 2019 · 0 comments · Fixed by #4476
Closed

Healthchecker init error when restart Kong with much worker process #4453

wuguangkuo opened this issue Mar 29, 2019 · 0 comments · Fixed by #4476

Comments

@wuguangkuo
Copy link

wuguangkuo commented Mar 29, 2019

Summary

Healthchecker init error when restart Kong with much worker process

Steps To Reproduce

1.Configure an upstream object with some targets
2.Ensure the server create much nginx worker process:nginx_worker_processes=auto or a value between about 20~40
3.restart Kong
4.the error.log print some exception stack

Additional Details & Logs

  • Kong version (0.14.0)
  • Kong error logs (<KONG_PREFIX>/logs/error.log)

2019/03/28 19:23:57 [error] 112024#0: *95 [lua] events.lua:175: do_handlerlist(): worker-events: event callback failed; source=lua-resty-healthcheck [a2d0f38f-7be9-4b7d-bae6-9555d636be6d.api.com], event=healthy, pid=112004 error='/usr/local/share/lua/5.1/resty/healthcheck.lua:917: attempt to index field 'targets' (a nil value) stack traceback: /usr/local/share/lua/5.1/resty/healthcheck.lua:917: in function </usr/local/share/lua/5.1/resty/healthcheck.lua:915> [C]: in function 'xpcall' /usr/local/share/lua/5.1/resty/worker/events.lua:166: in function 'do_handlerlist' /usr/local/share/lua/5.1/resty/worker/events.lua:198: in function 'do_event_json' /usr/local/share/lua/5.1/resty/worker/events.lua:342: in function 'poll' /usr/local/share/lua/5.1/resty/worker/events.lua:362: in function </usr/local/share/lua/5.1/resty/worker/events.lua:356>', data={"port":8180,"ip":"172.16.100.32","hostname":"172.16.100.32"}, context: ngx.timer

  • Operating system: CentOS Linux release 7.4.1708 (Core)
hishamhm pushed a commit to Kong/lua-resty-healthcheck that referenced this issue Apr 4, 2019
This fixes an issue observed in Kong: when restarting the kong process, the concurrent worker processes load targets, and trigger other workers's event callback. When callbacks hit before the target list is ready, the event_handler method error would happen at line 925: `the targets is nil` when indexing its `ip` field.

See Kong/kong#4453
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant