-
-
Notifications
You must be signed in to change notification settings - Fork 49
/
index.md
396 lines (283 loc) · 12 KB
/
index.md
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
---
toc:
toc_depth: 3
---
fakeredis: A python implementation of redis server
=================================================
FakeRedis is a pure-Python implementation of the Redis key-value store.
It enables running tests requiring redis server without an actual server.
It provides enhanced versions of the redis-py Python bindings for Redis.
That provides the following added functionality: A built-in Redis server that is automatically installed, configured and
managed when the Redis bindings are used.
A single server shared by multiple programs or multiple independent servers.
All the servers provided by FakeRedis support all Redis functionality including advanced features such as RedisJson,
GeoCommands.
For a list of supported/unsupported redis commands, see [Supported commands][supported-commands].
## Installation
To install fakeredis-py, simply:
```bash
pip install fakeredis ## No additional modules support
pip install fakeredis[lua] ## Support for LUA scripts
pip install fakeredis[json] ## Support for RedisJSON commands
pip install fakeredis[probabilistic,json] ## Support for RedisJSON and BloomFilter/CuckooFilter/CountMinSketch commands
```
## How to Use
### Start a server on a thread
It is possible to start a server on a thread and use it as a connect to it as you would a real redis server.
```python
from threading import Thread
from fakeredis import TcpFakeServer
server_address = ("127.0.0.1", 6379)
server = TcpFakeServer(server_address, server_type="redis")
t = Thread(target=server.serve_forever, daemon=True)
t.start()
import redis
r = redis.Redis(host=server_address[0], port=server_address[1])
r.set("foo", "bar")
assert r.get("foo") == b"bar"
```
### Use as a pytest fixture
```python
import pytest
@pytest.fixture
def redis_client(request):
import fakeredis
redis_client = fakeredis.FakeRedis()
return redis_client
```
### General usage
FakeRedis can imitate Redis server version 6.x or 7.x, [Valkey server](./valkey-support),
and [dragonfly server](./dragonfly-support). Redis version 7 is used by default.
The intent is for fakeredis to act as though you're talking to a real redis server.
It does this by storing the state internally. For example:
```pycon
>>> import fakeredis
>>> r = fakeredis.FakeStrictRedis(server_type="redis")
>>> r.set('foo', 'bar')
True
>>> r.get('foo')
'bar'
>>> r.lpush('bar', 1)
1
>>> r.lpush('bar', 2)
2
>>> r.lrange('bar', 0, -1)
[2, 1]
```
The state is stored in an instance of `FakeServer`. If one is not provided at
construction, a new instance is automatically created for you, but you can
explicitly create one to share state:
```pycon
>>> import fakeredis
>>> server = fakeredis.FakeServer()
>>> r1 = fakeredis.FakeStrictRedis(server=server)
>>> r1.set('foo', 'bar')
True
>>> r2 = fakeredis.FakeStrictRedis(server=server)
>>> r2.get('foo')
'bar'
>>> r2.set('bar', 'baz')
True
>>> r1.get('bar')
'baz'
>>> r2.get('bar')
'baz'
```
It is also possible to mock connection errors, so you can effectively test your error handling.
Set the connected attribute of the server to `False` after initialization.
```pycon
>>> import fakeredis
>>> server = fakeredis.FakeServer()
>>> server.connected = False
>>> r = fakeredis.FakeStrictRedis(server=server)
>>> r.set('foo', 'bar')
ConnectionError: FakeRedis is emulating a connection error.
>>> server.connected = True
>>> r.set('foo', 'bar')
True
```
Fakeredis implements the same interface as `redis-py`, the popular
redis client for python, and models the responses of redis 6.x or 7.x.
### async Redis
Async redis client is supported. Instead of using `fakeredis.FakeRedis`, use `fakeredis.aioredis.FakeRedis`.
```pycon
>>> from fakeredis import FakeAsyncRedis
>>> r1 = FakeAsyncRedis()
>>> await r1.set('foo', 'bar')
True
>>> await r1.get('foo')
'bar'
```
### Use to test django cache
Update your cache settings:
```python
from fakeredis import FakeConnection
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.redis.RedisCache',
'LOCATION': '...',
'OPTIONS': {
'connection_class': FakeConnection
}
}
}
```
For [django-redis][django-redis] library, use the following `OPTIONS`:
```
'OPTIONS': {
'CONNECTION_POOL_KWARGS': {'connection_class': FakeConnection},
}
```
You can use
django [`@override_settings` decorator][django-override-settings]
### Use to test django-rq
There is a need to override `django_rq.queues.get_redis_connection` with a method returning the same connection.
```python
import django_rq
# RQ
# Configuration to pretend there is a Redis service available.
# Set up the connection before RQ Django reads the settings.
# The connection must be the same because in fakeredis connections
# do not share the state. Therefore, we define a singleton object to reuse it.
def get_fake_connection(config: Dict[str, Any], strict: bool):
from fakeredis import FakeRedis, FakeStrictRedis
redis_cls = FakeStrictRedis if strict else FakeRedis
if "URL" in config:
return redis_cls.from_url(
config["URL"],
db=config.get("DB"),
)
return redis_cls(
host=config["HOST"],
port=config["PORT"],
db=config.get("DB", 0),
username=config.get("USERNAME", None),
password=config.get("PASSWORD"),
)
django_rq.queues.get_redis_connection = get_fake_connection
```
### Use to test FastAPI
See info on [this issue][fastapi-issue]
If you're using FastAPI dependency injection to provide a Redis connection,
then you can override that dependency for testing.
Your FastAPI application main.py:
```python
from typing import Annotated, Any, AsyncIterator
from redis import asyncio as redis
from fastapi import Depends, FastAPI
app = FastAPI()
async def get_redis() -> AsyncIterator[redis.Redis]:
# Code to handle creating a redis connection goes here, for example
async with redis.from_url("redis://localhost:6379") as client: # type: ignore[no-untyped-call]
yield client
@app.get("/")
async def root(redis_client: Annotated[redis.Redis, Depends(get_redis)]) -> Any:
# Code that does something with redis goes here, for example:
await redis_client.set("foo", "bar")
return {"redis_keys": await redis_client.keys()}
```
Assuming you use pytest-asyncio, your test file
(or you can put the fixtures in conftest.py as usual):
```python
from typing import AsyncIterator
from unittest import mock
import fakeredis
import httpx
import pytest
import pytest_asyncio
from redis import asyncio as redis
from main import app, get_redis
@pytest_asyncio.fixture
async def redis_client() -> AsyncIterator[redis.Redis]:
async with fakeredis.FakeAsyncRedis() as client:
yield client
@pytest_asyncio.fixture
async def app_client(redis_client: redis.Redis) -> AsyncIterator[httpx.AsyncClient]:
async def get_redis_override() -> redis.Redis:
return redis_client
transport = httpx.ASGITransport(app=app) # type: ignore[arg-type] # https://github.com/encode/httpx/issues/3111
async with httpx.AsyncClient(transport=transport, base_url="http://test") as app_client:
with mock.patch.dict(app.dependency_overrides, {get_redis: get_redis_override}):
yield app_client
@pytest.mark.asyncio
async def test_app(app_client: httpx.AsyncClient) -> None:
response = await app_client.get("/")
assert response.json()["redis_keys"] == ["foo"]
```
## Known Limitations
Apart from unimplemented commands, there are a number of cases where fakeredis won't give identical results to real
redis.
The following are differences that are unlikely to ever be fixed; there are also differences that are fixable (such as
commands that do not support all features) which should be filed as bugs in GitHub.
- Hyperloglogs are implemented using sets underneath. This means that the `type` command will return the wrong answer,
you can't use `get` to retrieve the encoded value, and counts will be slightly different (they will in fact be exact).
- When a command has multiple error conditions, such as operating on a key of the wrong type and an integer argument is
not well-formed, the choice of error to return may not match redis.
- The `incrbyfloat` and `hincrbyfloat` commands in redis use the C `long double` type, which typically has more
precision than Python's `float` type.
- Redis makes guarantees about the order in which clients blocked on blocking commands are woken up. Fakeredis does not
honor these guarantees.
- Where redis contains bugs, fakeredis generally does not try to provide exact bug compatibility. It's not practical for
fakeredis to try to match the set of bugs in your specific version of redis.
- There are a number of cases where the behavior of redis is undefined, such as the order of elements returned by set
and hash commands. Fakeredis will generally not produce the same results, and in Python versions before 3.6 may
produce different results each time the process is re-run.
- SCAN/ZSCAN/HSCAN/SSCAN will not necessarily iterate all items if items are deleted or renamed during iteration. They
also won't necessarily iterate in the same chunk sizes or the same order as redis. This is aligned with redis behavior
as can be seen in tests `test_scan_delete_key_while_scanning_should_not_returns_it_in_scan`.
- DUMP/RESTORE will not return or expect data in the RDB format. Instead, the `pickle` module is used to mimic an opaque
and non-standard format. **WARNING**: Do not use RESTORE with untrusted data, as a malicious pickle can execute
arbitrary code.
## Local development environment
To ensure parity with the real redis, there are a set of integration tests that mirror the unittests. For every unittest
that is written, the same test is run against a real redis instance using a real redis-py client instance. To run these
tests, you must have a redis server running on localhost, port 6379 (the default settings). **WARNING**: the tests will
completely wipe your database!
First install poetry if you don't have it, and then install all the dependencies:
```bash
pip install poetry
poetry install
```
To run all the tests:
```bash
poetry run pytest -v
```
If you only want to run tests against fake redis, without a real redis::
```bash
poetry run pytest -m fake
```
Because this module is attempting to provide the same interface as `redis-py`, the python bindings to redis, a
reasonable way to test this to take each unittest and run it against a real redis server.
Fakeredis and the real redis server should give the same result.
To run tests against a real redis instance instead:
```bash
poetry run pytest -m real
```
If redis is not running, and you try to run tests against a real redis server, these tests will have a result of 's' for
skipped.
There are some tests that test redis blocking operations that are somewhat slow.
If you want to skip these tests during day-to-day development, they have all been tagged as 'slow' so you can skip them
by running:
```bash
poetry run pytest -m "not slow"
```
## Contributing
Contributions are welcome. You can contribute in many ways:
- Report bugs you found.
- Check out issues with [`Help wanted`][help-wanted-issues] label.
- Implement commands which are not yet implemented. Follow
the [guide how to implement a new command][guide-implement-command].
- Write additional test cases. Follow the [guide how to write a test-case][guide-test-case].
Please follow coding standards listed in the [contributing guide][contributing].
## Sponsor
fakeredis-py is developed for free.
You can support this project by becoming a sponsor using [this link][sponsor].
[sponsor]:https://github.com/sponsors/cunla
[guide-implement-command]:./guides/implement-command
[contributing]:./about/contributing
[guide-test-case]:./guides/test-case
[help-wanted-issues]:https://github.com/cunla/fakeredis-py/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22
[supported-commands]:./supported-commands/
[fastapi-issue]:https://github.com/cunla/fakeredis-py/issues/292
[django-override-settings]:https://docs.djangoproject.com/en/4.1/topics/testing/tools/#django.test.override_settings
[django-redis]:https://github.com/jazzband/django-redis