Skip to content

Commit ebf8c02

Browse files
strawgatestrawgate
authored
Merge branch 'main' into claude/issue-105-20251026-0238
2 parents 612cc49 + d58fe24 commit ebf8c02

File tree

122 files changed

+3655
-970
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

122 files changed

+3655
-970
lines changed

‎.devcontainer/devcontainer.json‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
{
2+
"name": "py-key-value",
3+
"image": "ghcr.io/astral-sh/uv:python3.10-bookworm",
4+
"features": {
5+
"ghcr.io/devcontainers/features/node:1": {
6+
"version": "lts"
7+
},
8+
"ghcr.io/devcontainers/features/github-cli:1": {},
9+
"ghcr.io/devcontainers/features/git:1": {}
10+
},
11+
"runArgs": [
12+
"--network=host"
13+
],
14+
"mounts": [
15+
"source=/var/run/docker.sock,target=/var/run/docker.sock,type=bind"
16+
],
17+
"customizations": {
18+
"vscode": {
19+
"extensions": [
20+
"ms-python.python",
21+
"ms-python.vscode-pylance",
22+
"charliermarsh.ruff",
23+
"DavidAnson.vscode-markdownlint"
24+
],
25+
"settings": {
26+
"python.defaultInterpreterPath": "/usr/local/bin/python",
27+
"python.testing.pytestEnabled": true,
28+
"python.testing.unittestEnabled": false,
29+
"python.testing.pytestArgs": [
30+
"key-value",
31+
"--import-mode=importlib",
32+
"-vv"
33+
]
34+
}
35+
}
36+
},
37+
"postCreateCommand": "make sync",
38+
"remoteUser": "root"
39+
}

‎.gitignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,9 @@ __pycache__/
66
# C extensions
77
*.so
88

9+
# MacOS junk
10+
.DS_Store
11+
912
# Distribution / packaging
1013
.Python
1114
build/

‎AGENTS.md‎

Lines changed: 235 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,235 @@
1+
# AGENTS.md
2+
3+
This file provides guidelines and context for AI coding agents working on the
4+
py-key-value project. For human developers, see [DEVELOPING.md](DEVELOPING.md).
5+
6+
## Development Workflow
7+
8+
### Required Pre-commit Checks
9+
10+
All three checks must pass before committing:
11+
12+
1. `make lint` - Runs Ruff formatting and linting (Python + Markdown)
13+
2. `make typecheck` - Runs Basedpyright type checking
14+
3. `make codegen` - Regenerates sync library from async
15+
16+
Or run all three together:
17+
18+
```bash
19+
make precommit
20+
```
21+
22+
### Testing Requirements
23+
24+
- All new features require tests in both async and sync packages
25+
- Run `make test` to execute all test suites
26+
- Run `make test-aio` for async package tests only
27+
- Run `make test-sync` for sync package tests only
28+
- Test coverage should be maintained
29+
30+
## Architecture
31+
32+
### Async-First Development
33+
34+
**This is a critical constraint**: Always modify the async package first.
35+
36+
- **Primary codebase**: `key-value/key-value-aio/` (async implementation)
37+
- **Generated codebase**: `key-value/key-value-sync/` (DO NOT EDIT DIRECTLY)
38+
- **Sync generation**: Run `make codegen` to generate sync from async
39+
40+
The sync library is automatically generated from the async library using
41+
`scripts/build_sync_library.py`. All changes must be made to the async
42+
package first, then regenerated into the sync package.
43+
44+
### Monorepo Structure
45+
46+
```text
47+
key-value/
48+
├── key-value-aio/ # Primary async library
49+
├── key-value-sync/ # Generated sync library (DO NOT EDIT)
50+
├── key-value-shared/ # Shared utilities and types
51+
└── key-value-shared-test/ # Shared test utilities
52+
scripts/
53+
├── build_sync_library.py # Codegen script for sync library
54+
└── bump_versions.py # Version management script
55+
```
56+
57+
## Code Style & Conventions
58+
59+
### Python
60+
61+
- **Formatter/Linter**: Ruff (configured in `pyproject.toml`)
62+
- **Line length**: 140 characters
63+
- **Type checker**: Basedpyright (strict mode)
64+
- **Runtime type checking**: Beartype (can be disabled via
65+
`PY_KEY_VALUE_DISABLE_BEARTYPE=true`)
66+
- **Python version**: 3.10+ (sync codegen targets 3.10)
67+
68+
### Markdown
69+
70+
- **Linter**: markdownlint (`.markdownlint.jsonc`)
71+
- **Line length**: 80 characters (excluding code blocks and tables)
72+
73+
## Common Pitfalls
74+
75+
### ManagedEntry Wrapper Objects
76+
77+
Raw values are **NEVER** stored directly in backends. The `ManagedEntry` wrapper
78+
(from `key_value/shared/utils/managed_entry.py`) wraps values with metadata
79+
like TTL and creation timestamp, typically serialized to/from JSON.
80+
81+
When implementing or debugging stores, remember that what's stored is not
82+
the raw value but a `ManagedEntry` containing:
83+
84+
- The actual value
85+
- Creation timestamp
86+
- TTL metadata
87+
88+
### Python Version Compatibility
89+
90+
The sync codegen targets Python 3.10. Running the codegen script with a
91+
different Python version may produce unexpected results or compatibility
92+
issues. Use Python 3.10 when running `make codegen`.
93+
94+
### Optional Backend Dependencies
95+
96+
Store implementations have optional dependencies. Install extras as needed:
97+
98+
```bash
99+
pip install py-key-value-aio[redis] # Redis support
100+
pip install py-key-value-aio[dynamodb] # DynamoDB support
101+
pip install py-key-value-aio[mongodb] # MongoDB support
102+
# etc. - see README.md for full list
103+
```
104+
105+
### Sync Package is Generated
106+
107+
**Never edit files in `key-value/key-value-sync/` directly**. Any changes
108+
will be overwritten when `make codegen` runs. Always make changes in the
109+
async package and regenerate. Always run `make codegen` after making changes
110+
to the async package. You will need to include the generated code in your pull
111+
request. Nobody will generate it for you. This also means pull requests will contain
112+
two copies of your changes, this is intentional!
113+
114+
## Make Commands Reference
115+
116+
| Command | Purpose |
117+
|---------|---------|
118+
| `make sync` | Install all dependencies |
119+
| `make install` | Alias for `make sync` |
120+
| `make lint` | Lint Python + Markdown |
121+
| `make typecheck` | Run Basedpyright type checking |
122+
| `make test` | Run all test suites |
123+
| `make test-aio` | Run async package tests |
124+
| `make test-sync` | Run sync package tests |
125+
| `make test-shared` | Run shared package tests |
126+
| `make codegen` | Generate sync library from async |
127+
| `make precommit` | Run lint + typecheck + codegen |
128+
| `make build` | Build all packages |
129+
130+
### Per-Project Commands
131+
132+
Add `PROJECT=<path>` to target a specific package:
133+
134+
```bash
135+
make lint PROJECT=key-value/key-value-aio
136+
make typecheck PROJECT=key-value/key-value-aio
137+
make test PROJECT=key-value/key-value-aio
138+
make build PROJECT=key-value/key-value-aio
139+
```
140+
141+
## Key Protocols and Interfaces
142+
143+
### AsyncKeyValue Protocol
144+
145+
The core async interface is `AsyncKeyValue` protocol from
146+
`key_value/aio/protocols/key_value.py`. All async stores implement this
147+
protocol, which defines:
148+
149+
- `get`, `get_many` - Retrieve values
150+
- `put`, `put_many` - Store values with optional TTL
151+
- `delete`, `delete_many` - Remove values
152+
- `ttl`, `ttl_many` - Get TTL information
153+
154+
### KeyValue Protocol (Sync)
155+
156+
The sync mirror is `KeyValue` from `key_value/sync/code_gen/protocols/key_value.py`,
157+
generated from the async protocol.
158+
159+
## Store Implementations
160+
161+
Stores are located in:
162+
163+
- Async: `key-value/key-value-aio/src/key_value/aio/stores/`
164+
- Sync: `key-value/key-value-sync/src/key_value/sync/code_gen/stores/`
165+
166+
Available backends include: DynamoDB, Elasticsearch, Memcached, Memory, Disk,
167+
MongoDB, Redis, RocksDB, Valkey, Vault, Windows Registry, Keyring, and more.
168+
169+
## Wrappers
170+
171+
Wrappers add functionality to stores and are located in:
172+
173+
- Async: `key-value/key-value-aio/src/key_value/aio/wrappers/`
174+
- Sync: `key-value/key-value-sync/src/key_value/sync/code_gen/wrappers/`
175+
176+
Wrappers include: Compression, Encryption, Logging, Statistics, Retry,
177+
Timeout, Cache, Prefix, TTL clamping, and more.
178+
179+
## Adapters
180+
181+
Adapters simplify store interactions but don't implement the protocol directly.
182+
Located in:
183+
184+
- Async: `key-value/key-value-aio/src/key_value/aio/adapters/`
185+
- Sync: `key-value/key-value-sync/src/key_value/sync/code_gen/adapters/`
186+
187+
Key adapters:
188+
189+
- `PydanticAdapter` - Type-safe Pydantic model storage
190+
- `RaiseOnMissingAdapter` - Raise exceptions for missing keys
191+
192+
## Development Environment
193+
194+
### Option 1: DevContainer (Recommended)
195+
196+
The repository includes a DevContainer configuration for consistent development
197+
environments. Open in VSCode and select "Reopen in Container" when prompted.
198+
199+
### Option 2: Local Development
200+
201+
Prerequisites:
202+
203+
- Python 3.10+
204+
- `uv` for dependency management
205+
- Node.js and npm for markdown linting
206+
207+
Setup:
208+
209+
```bash
210+
make sync
211+
```
212+
213+
## CI/CD
214+
215+
GitHub Actions workflows are in `.github/workflows/`:
216+
217+
- `test.yml` - Run tests across packages
218+
- `publish.yml` - Publish packages to PyPI
219+
- `claude-on-mention.yml` - Claude Code assistant (can make PRs)
220+
- `claude-on-open-label.yml` - Claude triage assistant (read-only analysis)
221+
222+
## Version Management
223+
224+
To bump versions across all packages:
225+
226+
```bash
227+
make bump-version VERSION=1.2.3 # Actual bump
228+
make bump-version-dry VERSION=1.2.3 # Dry run
229+
```
230+
231+
## Getting Help
232+
233+
- For human developer documentation, see [DEVELOPING.md](DEVELOPING.md)
234+
- For library usage documentation, see [README.md](README.md)
235+
- For package-specific information, see READMEs in each package directory

‎Makefile‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,7 @@ ifdef PROJECT
7575
@cd $(PROJECT) && uv sync --locked --group dev
7676
else
7777
@echo "Syncing all packages..."
78-
@uv sync --all-packages
78+
@uv sync --all-packages --group dev
7979
@npm install -g markdownlint-cli
8080
endif
8181

‎README.md‎

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -101,11 +101,12 @@ needs while keeping your framework code clean and backend-agnostic.
101101
- **No Live Objects**: Even when using the in-memory store, "live" objects are
102102
never returned from the store. You get a dictionary or a Pydantic model,
103103
hopefully a copy of what you stored, but never the same instance in memory.
104-
- **Dislike of Bear Bros**: Beartype is used for runtime type checking, it will
105-
report warnings if you get too cheeky with what you're passing around. If you
106-
are not a fan of beartype, you can disable it by setting the
107-
`PY_KEY_VALUE_DISABLE_BEARTYPE` environment variable to `true` or you can
108-
disable the warnings via the warn module.
104+
- **Dislike of Bear Bros**: Beartype is used for runtime type checking. Core
105+
protocol methods in store and wrapper implementations (put/get/delete/ttl
106+
and their batch variants) enforce types and will raise TypeError for
107+
violations. Other code produces warnings. You can disable all beartype
108+
checks by setting `PY_KEY_VALUE_DISABLE_BEARTYPE=true` or suppress warnings
109+
via the warnings module.
109110

110111
## Installation
111112

@@ -164,7 +165,7 @@ get(key: str, collection: str | None = None) -> dict[str, Any] | None:
164165
get_many(keys: list[str], collection: str | None = None) -> list[dict[str, Any] | None]:
165166

166167
put(key: str, value: dict[str, Any], collection: str | None = None, ttl: SupportsFloat | None = None) -> None:
167-
put_many(keys: list[str], values: Sequence[dict[str, Any]], collection: str | None = None, ttl: Sequence[SupportsFloat | None] | None = None) -> None:
168+
put_many(keys: list[str], values: Sequence[dict[str, Any]], collection: str | None = None, ttl: SupportsFloat | None = None) -> None:
168169

169170
delete(key: str, collection: str | None = None) -> bool:
170171
delete_many(keys: list[str], collection: str | None = None) -> int:
@@ -296,6 +297,7 @@ The following wrappers are available:
296297

297298
| Wrapper | Description | Example |
298299
|---------|---------------|-----|
300+
| CollectionRoutingWrapper | Route operations to different stores based on a collection name. | `CollectionRoutingWrapper(collection_map={"sessions": redis_store, "users": dynamo_store}, default_store=memory_store)` |
299301
| CompressionWrapper | Compress values before storing and decompress on retrieval. | `CompressionWrapper(key_value=memory_store, min_size_to_compress=0)` |
300302
| FernetEncryptionWrapper | Encrypt values before storing and decrypt on retrieval. | `FernetEncryptionWrapper(key_value=memory_store, source_material="your-source-material", salt="your-salt")` |
301303
| FallbackWrapper | Fallback to a secondary store when the primary store fails. | `FallbackWrapper(primary_key_value=memory_store, fallback_key_value=memory_store)` |
@@ -306,6 +308,7 @@ The following wrappers are available:
306308
| PrefixKeysWrapper | Prefix all keys with a given prefix. | `PrefixKeysWrapper(key_value=memory_store, prefix="users")` |
307309
| ReadOnlyWrapper | Prevent all write operations on the underlying store. | `ReadOnlyWrapper(key_value=memory_store, raise_on_write=True)` |
308310
| RetryWrapper | Retry failed operations with exponential backoff. | `RetryWrapper(key_value=memory_store, max_retries=3, initial_delay=0.1, max_delay=10.0, exponential_base=2.0)` |
311+
| RoutingWrapper | Route operations to different stores based on a routing function. | `RoutingWrapper(routing_function=lambda collection: redis_store if collection == "sessions" else dynamo_store, default_store=memory_store)` |
309312
| SingleCollectionWrapper | Wrap a store to only use a single collection. | `SingleCollectionWrapper(key_value=memory_store, single_collection="users")` |
310313
| TTLClampWrapper | Clamp the TTL to a given range. | `TTLClampWrapper(key_value=memory_store, min_ttl=60, max_ttl=3600)` |
311314
| StatisticsWrapper | Track operation statistics for the store. | `StatisticsWrapper(key_value=memory_store)` |

‎key-value/key-value-aio/pyproject.toml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,7 @@ addopts = [
5656
"--inline-snapshot=disable",
5757
"-n=auto",
5858
"--dist=loadfile",
59+
"--maxfail=5"
5960
]
6061
markers = [
6162
"skip_on_ci: Skip running the test when running on CI",

0 commit comments

Comments
 (0)