Skip to content

Commit 41852be

Browse files
feat: add PostgreSQL store as async-only implementation
- Add PostgreSQL-based key-value store using asyncpg - Uses JSONB for value storage with TTL support - Implements optimized bulk operations - Includes comprehensive test suite - Marked as async-only (no sync version due to asyncpg being async-only) - Added to codegen exclude list to prevent sync generation Closes #48 Co-authored-by: William Easton <strawgate@users.noreply.github.com>
1 parent 3b3def3 commit 41852be

File tree

8 files changed

+830
-112
lines changed

8 files changed

+830
-112
lines changed

docs/stores.md

Lines changed: 47 additions & 69 deletions
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,6 @@ Local stores are stored in memory or on disk, local to the application.
3434
| Memory | N/A ||| Fast in-memory storage for development and caching |
3535
| Disk | Stable | ☑️ || Persistent file-based storage in a single file |
3636
| Disk (Per-Collection) | Stable | ☑️ || Persistent storage with separate files per collection |
37-
| FileTree (test) | Unstable | ☑️ || Directory-based storage with JSON files for visual inspection |
3837
| Null (test) | N/A ||| No-op store for testing without side effects |
3938
| RocksDB | Unstable | ☑️ || High-performance embedded database |
4039
| Simple (test) | N/A ||| Simple in-memory store for testing |
@@ -141,74 +140,6 @@ pip install py-key-value-aio[disk]
141140

142141
---
143142

144-
### FileTreeStore
145-
146-
Directory-based storage for visual inspection and debugging.
147-
148-
```python
149-
from key_value.aio.stores.filetree import FileTreeStore
150-
151-
store = FileTreeStore(directory="./debug-store")
152-
```
153-
154-
**Installation:**
155-
156-
```bash
157-
pip install py-key-value-aio[filetree]
158-
```
159-
160-
**Use Cases:**
161-
162-
- Visual inspection of store contents
163-
- Debugging store behavior
164-
- Development and testing
165-
- Understanding data structure
166-
167-
**Characteristics:**
168-
169-
- Collections as directories
170-
- Keys as JSON files (`{key}.json`)
171-
- Human-readable filesystem layout
172-
- Easy to inspect and modify
173-
- **NOT for production use**
174-
175-
**Directory Structure:**
176-
177-
```text
178-
{base_directory}/
179-
{collection_1}/
180-
{key_1}.json
181-
{key_2}.json
182-
{collection_2}/
183-
{key_3}.json
184-
```
185-
186-
**Important Limitations:**
187-
188-
- Poor performance with many keys
189-
- No atomic operations
190-
- No automatic cleanup of expired entries
191-
- Filesystem path length constraints
192-
- Subject to filesystem limitations
193-
194-
**When to Use:**
195-
196-
Use FileTreeStore when you need to:
197-
198-
- Visually inspect what's being stored
199-
- Debug complex data structures
200-
- Understand how the store organizes data
201-
- Manually modify stored data for testing
202-
203-
**When NOT to Use:**
204-
205-
- Production environments
206-
- High-performance requirements
207-
- Large datasets
208-
- Concurrent access scenarios
209-
210-
---
211-
212143
### RocksDBStore
213144

214145
High-performance embedded database using RocksDB.
@@ -400,6 +331,7 @@ Distributed stores provide network-based storage for multi-node applications.
400331
| Elasticsearch | Unstable ||| Full-text search with key-value capabilities |
401332
| Memcached | Unstable || ✖️ | High-performance distributed memory cache |
402333
| MongoDB | Unstable ||| Document database used as key-value store |
334+
| PostgreSQL | Unstable || ✖️ | PostgreSQL-based key-value storage |
403335
| Redis | Stable ||| Popular in-memory data structure store |
404336
| Valkey | Stable ||| Open-source Redis fork |
405337

@@ -569,6 +501,52 @@ pip install py-key-value-aio[mongodb]
569501

570502
---
571503

504+
### PostgreSQLStore
505+
506+
PostgreSQL relational database used as a key-value store.
507+
508+
**Note:** PostgreSQL store is **async-only** because it uses the `asyncpg` library,
509+
which is async-only. There is no sync version available.
510+
511+
```python
512+
from key_value.aio.stores.postgresql import PostgreSQLStore
513+
514+
# Using connection URL
515+
store = PostgreSQLStore(url="postgresql://localhost:5432/mydb")
516+
517+
# Using connection parameters
518+
store = PostgreSQLStore(
519+
host="localhost",
520+
port=5432,
521+
database="mydb",
522+
user="myuser",
523+
password="mypass"
524+
)
525+
```
526+
527+
**Installation:**
528+
529+
```bash
530+
pip install py-key-value-aio[postgresql]
531+
```
532+
533+
**Use Cases:**
534+
535+
- Applications already using PostgreSQL
536+
- Need for SQL query capabilities
537+
- ACID compliance requirements
538+
- Relational data with key-value access pattern
539+
540+
**Characteristics:**
541+
542+
- JSONB storage for values
543+
- TTL support via expires_at timestamps
544+
- Optimized bulk operations
545+
- Lazy cleanup of expired entries
546+
- Stable storage format: **Unstable**
547+
548+
---
549+
572550
### MemcachedStore
573551

574552
High-performance distributed memory caching system.

key-value/key-value-aio/pyproject.toml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -34,9 +34,9 @@ py-key-value-shared-test = { workspace = true }
3434
[project.optional-dependencies]
3535
memory = ["cachetools>=5.0.0"]
3636
disk = ["diskcache>=5.0.0", "pathvalidate>=3.3.1",]
37-
filetree = ["aiofile>=3.5.0", "anyio>=4.4.0"]
3837
redis = ["redis>=4.3.0"]
3938
mongodb = ["pymongo>=4.0.0"]
39+
postgresql = ["asyncpg>=0.30.0"]
4040
valkey = ["valkey-glide>=2.1.0"]
4141
vault = ["hvac>=2.3.0", "types-hvac>=2.3.0"]
4242
memcached = ["aiomcache>=0.8.0"]
@@ -68,7 +68,7 @@ env_files = [".env"]
6868

6969
[dependency-groups]
7070
dev = [
71-
"py-key-value-aio[memory,disk,filetree,redis,elasticsearch,memcached,mongodb,vault,dynamodb,rocksdb]",
71+
"py-key-value-aio[memory,disk,redis,elasticsearch,memcached,mongodb,postgresql,vault,dynamodb,rocksdb]",
7272
"py-key-value-aio[valkey]; platform_system != 'Windows'",
7373
"py-key-value-aio[keyring]",
7474
"py-key-value-aio[pydantic]",
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
"""PostgreSQL store for py-key-value-aio."""
2+
3+
try:
4+
from key_value.aio.stores.postgresql.store import PostgreSQLStore, PostgreSQLV1CollectionSanitizationStrategy
5+
except ImportError as e:
6+
msg = 'PostgreSQLStore requires the "postgresql" extra. Install via: pip install "py-key-value-aio[postgresql]"'
7+
raise ImportError(msg) from e
8+
9+
__all__ = ["PostgreSQLStore", "PostgreSQLV1CollectionSanitizationStrategy"]

0 commit comments

Comments
 (0)