feat: Graceful server shutdown and enforce only one running instance of Phoenix at a time #155
Conversation
nate-mar
changed the title
src/phoenix/services.py
Outdated
| if len(os.listdir(server.get_pids_path())) > 0: | ||
| # Currently, only one instance of Phoenix can be running at any given time. | ||
| # Support for multiple concurrently running instances may be supported in the future. | ||
| logger.warning("Existing running Phoenix instance detected! Shutting it down...") |
There was a problem hiding this comment.
Do we want to warn and exit instead and force the user to explicitly ask to kill the existing instance?
| @@ -37,13 +45,24 @@ def start(self) -> None: | |||
| ) | |||
|
|
|||
| def stop(self) -> None: | |||
| """Stops the service.""" | |||
| """Gracefully stops the service.""" | |||
There was a problem hiding this comment.
Is graceful shutdown necessary? If a user kills any existing Server instance while launching the app, any cleanup will not be performed. However, I don't see a lot of actual clean up being done at the moment.
src/phoenix/server/__init__.py
Outdated
| def _get_temp_path() -> str: | ||
| """Get path to directory in which to store temp phoenix server files.""" | ||
| return os.path.join(tempfile.gettempdir(), ".arize-phoenix") | ||
|
|
||
|
|
||
| def get_pids_path() -> str: | ||
| """Get path to directory in which to store temp phoenix instance pid files. | ||
| This directory is used to track any currently running instances of Arize Phoenix | ||
| on the host machine. The directory will be created if it does not exist. | ||
| """ | ||
| path = os.path.join(_get_temp_path(), "pids") | ||
| try: | ||
| os.makedirs(path) | ||
| except OSError as e: | ||
| if e.errno == errno.EEXIST: | ||
| pass | ||
| else: | ||
| raise | ||
| else: | ||
| os.chmod(path, 0o777) | ||
| return path |
There was a problem hiding this comment.
There's a config module here: https://github.com/Arize-ai/phoenix/blob/main/src/phoenix/config.py
I think we should centralize the paths here - should we move dataset files into temp too?
There was a problem hiding this comment.
I actually originally had it in there, but I moved it out because I thought it wouldn't generally be something configurable, and would only be relevant to the server. Cool with moving it back, if you just want to centralize.
| raise | ||
|
|
||
|
|
||
| def _get_pid_file() -> str: |
There was a problem hiding this comment.
nit: maybe name it as _pid_file_path? file implies content to me
There was a problem hiding this comment.
ah shoot, I thought I had changed this. will do it in a follow up
src/phoenix/server/main.py
Outdated
| def _write_pid_file(): | ||
| with open(_get_pid_file(), "w") as outfile: | ||
| pass | ||
|
|
||
|
|
||
| def _remove_pid_file() -> None: | ||
| try: | ||
| os.unlink(_get_pid_file()) | ||
| except OSError as e: | ||
| if e.errno == errno.ENOENT: | ||
| # If the pid file doesn't exist, ignore and continue on since | ||
| # we are already in the desired end state; This should not happen | ||
| pass | ||
| else: | ||
| raise |
There was a problem hiding this comment.
thought: it sorta feels like the processes should be managed / encapsulated in the session and that the session tears down the processes?
There was a problem hiding this comment.
Yeah we can revisit this. The process is associated with the server, rather than the session; so not entirely sure what this would entail.
Resolves #97