Skip to content

Refactor async client connection-handling #280

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 26 commits into from
Feb 20, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
6259604
WIP
abrookins Feb 13, 2025
7907674
Merge branch 'main' into fix/RAAE-595-async-client-tweaks
abrookins Feb 13, 2025
af1c2d0
WIP
abrookins Feb 13, 2025
c2bf280
More cleanup
abrookins Feb 13, 2025
c199968
Format, lint
abrookins Feb 13, 2025
d27c36e
ignore our own deprecation warnings
abrookins Feb 13, 2025
49bf7c4
Put key expiration behind an interface
abrookins Feb 14, 2025
b5af151
todo: fix the deprecation warnings instead of hiding
abrookins Feb 14, 2025
ac42d74
normalize sync and async index interfaces
abrookins Feb 14, 2025
9114941
retain support for sync -> async conversion in SemanticCache
abrookins Feb 14, 2025
2b171d0
Merge branch 'main' into fix/RAAE-595-async-client-tweaks
abrookins Feb 14, 2025
3eb1d43
add finalizers, explicit disconnect methods, ctx mgrs
abrookins Feb 15, 2025
e9470d3
Rejigger the finalizers and more
abrookins Feb 20, 2025
aaa2091
lint
abrookins Feb 20, 2025
8f8642f
Scale back module validation to match existing behavior
abrookins Feb 20, 2025
3198840
remove required_modules param from `from_existing`
abrookins Feb 20, 2025
cd37fe9
old man yells at mypy emoji
abrookins Feb 20, 2025
8dab1e4
Update docs
abrookins Feb 20, 2025
b33c543
Merge branch 'main' into fix/RAAE-595-async-client-tweaks
abrookins Feb 20, 2025
33cf7c5
Use default required mods unless needing introspection
abrookins Feb 20, 2025
35c2277
finish the job on module variable shuffling
abrookins Feb 20, 2025
a9b014b
use logger
tylerhutcherson Feb 20, 2025
fd2b417
remove unnecessary comment
abrookins Feb 20, 2025
3abd8ee
remove print output
abrookins Feb 20, 2025
7c22034
Merge branch 'main' into fix/RAAE-595-async-client-tweaks
tylerhutcherson Feb 20, 2025
2ec7402
reference redis_url as a kwarg
tylerhutcherson Feb 20, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 5 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,19 +121,18 @@ Choose from multiple Redis deployment options:
})
```

2. [Create a SearchIndex](https://docs.redisvl.com/en/stable/user_guide/01_getting_started.html#create-a-searchindex) class with an input schema and client connection in order to perform admin and search operations on your index in Redis:
2. [Create a SearchIndex](https://docs.redisvl.com/en/stable/user_guide/01_getting_started.html#create-a-searchindex) class with an input schema to perform admin and search operations on your index in Redis:
```python
from redis import Redis
from redisvl.index import SearchIndex

# Establish Redis connection and define index
client = Redis.from_url("redis://localhost:6379")
index = SearchIndex(schema, client)
# Define the index
index = SearchIndex(schema, redis_url="redis://localhost:6379")

# Create the index in Redis
index.create()
```
> Async compliant search index class also available: [AsyncSearchIndex](https://docs.redisvl.com/en/stable/api/searchindex.html#redisvl.index.AsyncSearchIndex).
> An async-compatible index class also available: [AsyncSearchIndex](https://docs.redisvl.com/en/stable/api/searchindex.html#redisvl.index.AsyncSearchIndex).

3. [Load](https://docs.redisvl.com/en/stable/user_guide/01_getting_started.html#load-data-to-searchindex)
and [fetch](https://docs.redisvl.com/en/stable/user_guide/01_getting_started.html#fetch-an-object-from-redis) data to/from your Redis instance:
Expand Down Expand Up @@ -346,7 +345,7 @@ Commands:
stats Obtain statistics about an index
```

> Read more about [using the CLI](https://docs.redisvl.com/en/stable/user_guide/cli.html).
> Read more about [using the CLI](https://docs.redisvl.com/en/latest/overview/cli.html).

## 🚀 Why RedisVL?

Expand Down
100 changes: 56 additions & 44 deletions docs/user_guide/01_getting_started.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
Expand Down Expand Up @@ -126,7 +126,7 @@
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
Expand Down Expand Up @@ -178,7 +178,7 @@
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -195,7 +195,7 @@
"Now we also need to facilitate a Redis connection. There are a few ways to do this:\n",
"\n",
"- Create & manage your own client connection (recommended)\n",
"- Provide a simple Redis URL and let RedisVL connect on your behalf"
"- Provide a Redis URL and let RedisVL connect on your behalf (by default, it will connect to \"redis://localhost:6379\")"
]
},
{
Expand All @@ -209,7 +209,7 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": 6,
"metadata": {},
"outputs": [
{
Expand All @@ -227,9 +227,13 @@
"from redis import Redis\n",
"\n",
"client = Redis.from_url(\"redis://localhost:6379\")\n",
"index = SearchIndex.from_dict(schema, redis_client=client)\n",
"\n",
"index.set_client(client)\n",
"# optionally provide an async Redis client object to enable async index operations"
"# alternatively, provide an async Redis client object to enable async index operations\n",
"# from redis.asyncio import Redis\n",
"# from redisvl.index import AsyncSearchIndex\n",
"# client = Redis.from_url(\"redis://localhost:6379\")\n",
"# index = AsyncSearchIndex.from_dict(schema, redis_client=client)\n"
]
},
{
Expand All @@ -243,7 +247,7 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": 7,
"metadata": {},
"outputs": [
{
Expand All @@ -258,8 +262,10 @@
}
],
"source": [
"index.connect(\"redis://localhost:6379\")\n",
"# optionally use an async client by passing use_async=True"
"index = SearchIndex.from_dict(schema, redis_url=\"redis://localhost:6379\")\n",
"\n",
"# If you don't specify a client or Redis URL, the index will attempt to\n",
"# connect to Redis at the default address (\"redis://localhost:6379\")."
]
},
{
Expand All @@ -273,7 +279,7 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -297,7 +303,7 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": 9,
"metadata": {},
"outputs": [
{
Expand All @@ -315,7 +321,7 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": 10,
"metadata": {},
"outputs": [
{
Expand Down Expand Up @@ -358,7 +364,7 @@
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": 11,
"metadata": {},
"outputs": [
{
Expand Down Expand Up @@ -392,7 +398,7 @@
},
{
"cell_type": "code",
"execution_count": 10,
"execution_count": 12,
"metadata": {},
"outputs": [
{
Expand Down Expand Up @@ -429,7 +435,7 @@
},
{
"cell_type": "code",
"execution_count": 11,
"execution_count": 13,
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -454,13 +460,20 @@
},
{
"cell_type": "code",
"execution_count": 12,
"execution_count": 14,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"*=>[KNN 3 @user_embedding $vector AS vector_distance] RETURN 6 user age job credit_score vector_distance vector_distance SORTBY vector_distance ASC DIALECT 2 LIMIT 0 3\n"
]
},
{
"data": {
"text/html": [
"<table><tr><th>vector_distance</th><th>user</th><th>age</th><th>job</th><th>credit_score</th></tr><tr><td>0</td><td>john</td><td>1</td><td>engineer</td><td>high</td></tr><tr><td>0</td><td>mary</td><td>2</td><td>doctor</td><td>low</td></tr><tr><td>0.0566299557686</td><td>tyler</td><td>9</td><td>engineer</td><td>high</td></tr></table>"
"table><tr><th>vector_distance</th><th>user</th><th>age</th><th>job</th><th>credit_score</th></tr><tr><td>0</td><td>john</td><td>1</td><td>engineer</td><td>high</td></tr><tr><td>0</td><td>mary</td><td>2</td><td>doctor</td><td>low</td></tr><tr><td>0.0566299557686</td><td>tyler</td><td>9</td><td>engineer</td><td>high</td></tr></table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
Expand All @@ -487,7 +500,7 @@
},
{
"cell_type": "code",
"execution_count": 13,
"execution_count": 15,
"metadata": {},
"outputs": [
{
Expand Down Expand Up @@ -537,13 +550,12 @@
"\n",
"client = Redis.from_url(\"redis://localhost:6379\")\n",
"\n",
"index = AsyncSearchIndex.from_dict(schema)\n",
"await index.set_client(client)"
"index = AsyncSearchIndex.from_dict(schema, redis_client=client)"
]
},
{
"cell_type": "code",
"execution_count": 15,
"execution_count": 16,
"metadata": {},
"outputs": [
{
Expand Down Expand Up @@ -584,7 +596,7 @@
},
{
"cell_type": "code",
"execution_count": 16,
"execution_count": 17,
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -609,14 +621,14 @@
},
{
"cell_type": "code",
"execution_count": 16,
"execution_count": 18,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"11:53:25 redisvl.index.index INFO Index already exists, overwriting.\n"
"11:28:32 redisvl.index.index INFO Index already exists, overwriting.\n"
]
}
],
Expand All @@ -627,13 +639,13 @@
},
{
"cell_type": "code",
"execution_count": 17,
"execution_count": 19,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<table><tr><th>vector_distance</th><th>user</th><th>age</th><th>job</th><th>credit_score</th></tr><tr><td>0</td><td>john</td><td>1</td><td>engineer</td><td>high</td></tr><tr><td>0</td><td>mary</td><td>2</td><td>doctor</td><td>low</td></tr><tr><td>0.0566299557686</td><td>tyler</td><td>9</td><td>engineer</td><td>high</td></tr></table>"
"<table><tr><th>vector_distance</th><th>user</th><th>age</th><th>job</th><th>credit_score</th></tr><tr><td>0</td><td>mary</td><td>2</td><td>doctor</td><td>low</td></tr><tr><td>0</td><td>john</td><td>1</td><td>engineer</td><td>high</td></tr><tr><td>0.0566299557686</td><td>tyler</td><td>9</td><td>engineer</td><td>high</td></tr></table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
Expand All @@ -659,7 +671,7 @@
},
{
"cell_type": "code",
"execution_count": 18,
"execution_count": 20,
"metadata": {},
"outputs": [
{
Expand All @@ -677,19 +689,19 @@
"│ num_records │ 22 │\n",
"│ percent_indexed │ 1 │\n",
"│ hash_indexing_failures │ 0 │\n",
"│ number_of_uses │ 5 │\n",
"│ bytes_per_record_avg │ 50.9091 │\n",
"│ number_of_uses │ 2 │\n",
"│ bytes_per_record_avg │ 47.8 │\n",
"│ doc_table_size_mb │ 0.000423431 │\n",
"│ inverted_sz_mb │ 0.00106812 │\n",
"│ inverted_sz_mb │ 0.000911713 │\n",
"│ key_table_size_mb │ 0.000165939 │\n",
"│ offset_bits_per_record_avg │ 8 │\n",
"│ offset_vectors_sz_mb │ 5.72205e-06 │\n",
"│ offsets_per_term_avg │ 0.272727 │\n",
"│ records_per_doc_avg │ 5.5 │\n",
"│ offset_bits_per_record_avg │ nan │\n",
"│ offset_vectors_sz_mb │ 0 │\n",
"│ offsets_per_term_avg │ 0 │\n",
"│ records_per_doc_avg │ 5 │\n",
"│ sortable_values_size_mb │ 0 │\n",
"│ total_indexing_time │ 0.197 │\n",
"│ total_inverted_index_blocks │ 12 │\n",
"│ vector_index_sz_mb │ 0.0201416 │\n",
"│ total_indexing_time │ 0.239 │\n",
"│ total_inverted_index_blocks │ 11 │\n",
"│ vector_index_sz_mb │ 0.235603 │\n",
"╰─────────────────────────────┴─────────────╯\n"
]
}
Expand Down Expand Up @@ -718,7 +730,7 @@
},
{
"cell_type": "code",
"execution_count": 19,
"execution_count": 21,
"metadata": {},
"outputs": [
{
Expand All @@ -727,7 +739,7 @@
"4"
]
},
"execution_count": 19,
"execution_count": 21,
"metadata": {},
"output_type": "execute_result"
}
Expand All @@ -739,7 +751,7 @@
},
{
"cell_type": "code",
"execution_count": 20,
"execution_count": 22,
"metadata": {},
"outputs": [
{
Expand All @@ -748,7 +760,7 @@
"True"
]
},
"execution_count": 20,
"execution_count": 22,
"metadata": {},
"output_type": "execute_result"
}
Expand All @@ -760,7 +772,7 @@
},
{
"cell_type": "code",
"execution_count": 21,
"execution_count": 23,
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -771,7 +783,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"display_name": "env",
"language": "python",
"name": "python3"
},
Expand Down
Loading