Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: better reload options #362

Merged
merged 12 commits into from
Aug 22, 2024
15 changes: 14 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ A Rust HTTP server for Python applications.
The main reasons behind Granian design are:

- Have a single, correct HTTP implementation, supporting versions 1, 2 (and eventually 3)
- Provide a single package for several platforms
- Provide a single package for several platforms
- Avoid the usual Gunicorn + uvicorn + http-tools dependency composition on unix systems
- Provide stable [performance](https://github.com/emmett-framework/granian/blob/master/benchmarks/README.md) when compared to existing alternatives

Expand Down Expand Up @@ -192,6 +192,19 @@ Options:
changes (requires granian[reload] extra)
[env var: GRANIAN_RELOAD; default:
(disabled)]
--reload-paths PATH Paths to watch for changes [env var:
GRANIAN_RELOAD_PATHS; default: (Working
directory)]
--reload-ignore-dirs TEXT Names of directories to ignore changes for.
Extends the default list of directories to
ignore in watchfiles' default filter [env
var: GRANIAN_RELOAD_IGNORE_DIRS]
--reload-ignore-patterns TEXT Path patterns (regex) to ignore changes for.
Extends the default list of patterns to
ignore in watchfiles' default filter [env
var: GRANIAN_RELOAD_IGNORE_PATTERNS]
--reload-ignore-paths PATH Absolute paths to ignore changes for [env
var: GRANIAN_RELOAD_IGNORE_PATHS]
--process-name TEXT Set a custom name for processes (requires
granian[pname] extra) [env var:
GRANIAN_PROCESS_NAME]
Expand Down
39 changes: 38 additions & 1 deletion granian/cli.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import json
import pathlib
from enum import Enum
from typing import Any, Callable, Optional, Type, TypeVar, Union
from typing import Any, Callable, List, Optional, Type, TypeVar, Union

import click

Expand Down Expand Up @@ -194,6 +194,35 @@ def option(*param_decls: str, cls: Optional[Type[click.Option]] = None, **attrs:
default=False,
help="Enable auto reload on application's files changes (requires granian[reload] extra)",
)
@option(
'--reload-paths',
type=click.Path(exists=True, file_okay=True, dir_okay=True, readable=True, path_type=pathlib.Path),
help='Paths to watch for changes',
show_default='Working directory',
multiple=True,
)
@option(
'--reload-ignore-dirs',
help=(
'Names of directories to ignore changes for. '
"Extends the default list of directories to ignore in watchfiles' default filter"
),
multiple=True,
)
@option(
'--reload-ignore-patterns',
help=(
'Path patterns (regex) to ignore changes for. '
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quick note because I stumbled over this while testing the feature.

I had also assumed that this will regex match on the full path of something that triggered the reload. However, this only matches the last part of the path, meaning the exact file/dir that triggered the reload, not the entire path.

https://watchfiles.helpmanual.io/api/filters/#watchfiles.DefaultFilter.ignore_entity_patterns

"Entity" isn't a much better description 😛 but "Path" might be suggesting a different behaviour altogether. WDYT?

I want to avoid issues where users report this to be broken.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I missed that. Would Relative path patterns make sense to you?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the late answer.

I think being fully explicit could help.

Suggested change
'Path patterns (regex) to ignore changes for. '
'File/directory name patterns (regex) to ignore changes for. '

WDYT?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks perfect. Can you open a PR for this? otherwise I'll do it by myself later

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No worries, will do!

"Extends the default list of patterns to ignore in watchfiles' default filter"
),
multiple=True,
)
@option(
'--reload-ignore-paths',
type=click.Path(exists=False, path_type=pathlib.Path),
help='Absolute paths to ignore changes for',
multiple=True,
)
@option(
'--process-name',
help='Set a custom name for processes (requires granian[pname] extra)',
Expand Down Expand Up @@ -242,6 +271,10 @@ def cli(
respawn_failed_workers: bool,
respawn_interval: float,
reload: bool,
reload_paths: Optional[List[pathlib.Path]],
reload_ignore_dirs: Optional[List[str]],
reload_ignore_patterns: Optional[List[str]],
reload_ignore_paths: Optional[List[pathlib.Path]],
process_name: Optional[str],
pid_file: Optional[pathlib.Path],
) -> None:
Expand Down Expand Up @@ -294,6 +327,10 @@ def cli(
respawn_failed_workers=respawn_failed_workers,
respawn_interval=respawn_interval,
reload=reload,
reload_paths=reload_paths,
reload_ignore_paths=reload_ignore_paths,
reload_ignore_dirs=reload_ignore_dirs,
reload_ignore_patterns=reload_ignore_patterns,
process_name=process_name,
pid_file=pid_file,
)
Expand Down
31 changes: 28 additions & 3 deletions granian/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
import time
from functools import partial
from pathlib import Path
from typing import Any, Callable, Dict, List, Optional, Tuple
from typing import Any, Callable, Dict, List, Optional, Sequence, Tuple, Type

from ._futures import future_watcher_wrapper
from ._granian import ASGIWorker, RSGIWorker, WSGIWorker
Expand Down Expand Up @@ -95,6 +95,11 @@ def __init__(
respawn_failed_workers: bool = False,
respawn_interval: float = 3.5,
reload: bool = False,
reload_paths: Optional[Sequence[Path]] = None,
reload_ignore_dirs: Optional[Sequence[str]] = None,
reload_ignore_patterns: Optional[Sequence[str]] = None,
reload_ignore_paths: Optional[Sequence[Path]] = None,
reload_filter: Optional[Type[watchfiles.BaseFilter]] = None,
process_name: Optional[str] = None,
pid_file: Optional[Path] = None,
):
Expand Down Expand Up @@ -129,6 +134,11 @@ def __init__(
self.respawn_failed_workers = respawn_failed_workers
self.reload_on_changes = reload
self.respawn_interval = respawn_interval
self.reload_paths = reload_paths or [Path.cwd()]
self.reload_ignore_paths = reload_ignore_paths or ()
self.reload_ignore_dirs = reload_ignore_dirs or ()
self.reload_ignore_patterns = reload_ignore_patterns or ()
self.reload_filter = reload_filter
self.process_name = process_name
self.pid_file = pid_file

Expand Down Expand Up @@ -580,13 +590,28 @@ def _serve_with_reloader(self, spawn_target, target_loader):
logger.error('Using --reload requires the granian[reload] extra')
sys.exit(1)

reload_path = Path.cwd()
# Use given or default filter rules
reload_filter = self.reload_filter or watchfiles.filters.DefaultFilter
# Extend `reload_filter` with explicit args
ignore_dirs = (*reload_filter.ignore_dirs, *self.reload_ignore_dirs)
ignore_entity_patterns = (
*reload_filter.ignore_entity_patterns,
*self.reload_ignore_patterns,
)
ignore_paths = (*reload_filter.ignore_paths, *self.reload_ignore_paths)
# Construct new filter
reload_filter = watchfiles.filters.DefaultFilter(
ignore_dirs=ignore_dirs, ignore_entity_patterns=ignore_entity_patterns, ignore_paths=ignore_paths
)

sock = self.startup(spawn_target, target_loader)

serve_loop = True
while serve_loop:
try:
for changes in watchfiles.watch(reload_path, stop_event=self.main_loop_interrupt):
for changes in watchfiles.watch(
*self.reload_paths, watch_filter=reload_filter, stop_event=self.main_loop_interrupt
):
logger.info('Changes detected, reloading workers..')
for change, file in changes:
logger.info(f'{change.raw_str().capitalize()}: {file}')
Expand Down
Loading