Skip to content

Commit 69407f6

Browse files
authored
Add LangCache wrapper (LLM cache extension) with configurable distance scale (#408)
## Summary Add a LangCache wrapper to RedisVL LLM cache extensions so users can leverage LangCache for semantic caching via the familiar `BaseLLMCache` interface. Also make the wrapper’s distance threshold semantics explicit and configurable so users can swap it out for an existing `SemanticCache` instance easily. ## Motivation - Use LangCache while keeping RedisVL’s developer experience - Avoid surprises by making threshold scale explicit ## What’s included - New `LangCacheWrapper` implementing `BaseLLMCache` (sync + async) that talks to the LangCache service: - Exact + semantic search toggles, TTL, and standard `store`/`check` APIs - Consistent result shape (including `vector_distance`) compatible with existing callers - Lazily imports LangCache and gives a friendly error if it's not installed - Configurable distance threshold scale via `distance_scale`: - `"normalized"`: interpret `distance_threshold` as 0–1 semantic distance; converts to/from similarity (`1 - d`) - `"redis"`: interpret `distance_threshold` as native COSINE [0–2]; converts using `norm_cosine_distance` / `denorm_cosine_distance` ## Usage ```python from redisvl.extensions.cache.llm.langcache import LangCacheWrapper cache = LangCacheWrapper( name="langcache", cache_id="<id>", api_key="<key>", distance_scale="normalized" # or "redis" for COSINE [0–2] ) cache.store(prompt="Hello?", response="Hi\!") hits = cache.check( prompt="Hello\!", distance_threshold=0.15, # interpreted per distance_scale ) ```
1 parent 3080562 commit 69407f6

File tree

12 files changed

+4851
-2943
lines changed

12 files changed

+4851
-2943
lines changed

pyproject.toml

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ name = "redisvl"
33
version = "0.11.0"
44
description = "Python client library and CLI for using Redis as a vector database"
55
authors = [{ name = "Redis Inc.", email = "applied.ai@redis.com" }]
6-
requires-python = ">=3.9,<3.14"
6+
requires-python = ">=3.9.2,<3.14"
77
readme = "README.md"
88
license = "MIT"
99
keywords = [
@@ -39,6 +39,7 @@ nltk = ["nltk>=3.8.1,<4"]
3939
cohere = ["cohere>=4.44"]
4040
voyageai = ["voyageai>=0.2.2"]
4141
sentence-transformers = ["sentence-transformers>=3.4.0,<4"]
42+
langcache = ["langcache>=0.9.0"]
4243
vertexai = [
4344
"google-cloud-aiplatform>=1.26,<2.0.0",
4445
"protobuf>=5.28.0,<6.0.0",
@@ -76,6 +77,7 @@ dev = [
7677
"testcontainers>=4.3.1,<5",
7778
"cryptography>=44.0.1 ; python_version > '3.9.1'",
7879
"codespell>=2.4.1,<3",
80+
"langcache>=0.9.0",
7981
]
8082
docs = [
8183
"sphinx>=4.4.0",

redisvl/cli/index.py

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,6 @@
88
from redisvl.redis.utils import convert_bytes, make_dict
99
from redisvl.schema.schema import IndexSchema
1010
from redisvl.utils.log import get_logger
11-
from redisvl.utils.utils import lazy_import
1211

1312
logger = get_logger("[RedisVL]")
1413

redisvl/extensions/cache/llm/__init__.py

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,11 +4,18 @@
44
This module provides LLM cache implementations for RedisVL.
55
"""
66

7+
from redisvl.extensions.cache.llm.langcache import LangCacheWrapper
78
from redisvl.extensions.cache.llm.schema import (
89
CacheEntry,
910
CacheHit,
1011
SemanticCacheIndexSchema,
1112
)
1213
from redisvl.extensions.cache.llm.semantic import SemanticCache
1314

14-
__all__ = ["SemanticCache", "CacheEntry", "CacheHit", "SemanticCacheIndexSchema"]
15+
__all__ = [
16+
"SemanticCache",
17+
"LangCacheWrapper",
18+
"CacheEntry",
19+
"CacheHit",
20+
"SemanticCacheIndexSchema",
21+
]

0 commit comments

Comments
 (0)