Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dettach duplicated signal handler from workers #2413

Merged
merged 1 commit into from
Sep 17, 2020

Conversation

adriangonz
Copy link
Contributor

What this PR does / why we need it:

Add a post_worker_init hook to dettach a signal handler from the Gunicorn worker processes to avoid calling join() when they exit. That signal handler is attached by multiprocessing on the root process, but gets inherited by the child workers because of how fork() works.

More details can be found in the following issue: benoitc/gunicorn#1391

Which issue(s) this PR fixes:

Fixes #2094

Special notes for your reviewer:
The signal handler should still remain on the root process.

Does this PR introduce a user-facing change?:

Fixed `can only join a child process` error when shutting down a Seldon deployment.

@adriangonz
Copy link
Contributor Author

/cc @RafalSkolasinski @axsaucedo

@seldondev
Copy link
Collaborator

Fri Sep 11 16:31:26 UTC 2020
The logs for [pr-build] [1] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2413/1.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2413 --build=1

@seldondev
Copy link
Collaborator

Fri Sep 11 16:31:29 UTC 2020
The logs for [lint] [2] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2413/2.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2413 --build=2

@adriangonz
Copy link
Contributor Author

/test integration

@adriangonz adriangonz changed the title Add post_worker_init hook to dettach multiprocessing signal handler Dettach duplicated signal handler from workers Sep 11, 2020
@seldondev
Copy link
Collaborator

Fri Sep 11 16:34:31 UTC 2020
The logs for [integration] [3] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2413/3.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2413 --build=3

@ukclivecox
Copy link
Contributor

/approve

@seldondev
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cliveseldon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@seldondev
Copy link
Collaborator

Thu Sep 17 09:10:06 UTC 2020
The logs for [lint] [5] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2413/5.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2413 --build=5

@seldondev
Copy link
Collaborator

Thu Sep 17 09:10:09 UTC 2020
The logs for [pr-build] [4] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2413/4.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2413 --build=4

@seldondev
Copy link
Collaborator

Thu Sep 17 12:58:45 UTC 2020
The logs for [pr-build] [6] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2413/6.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2413 --build=6

@seldondev
Copy link
Collaborator

Thu Sep 17 12:58:54 UTC 2020
The logs for [lint] [7] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2413/7.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2413 --build=7

@seldondev seldondev merged commit 3a79641 into SeldonIO:master Sep 17, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

can only join a child process
3 participants