Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding RESP3 tests support #2793

Merged
merged 48 commits into from
Jun 27, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
fc9fbc5
start cleaning
dvora-h Apr 30, 2023
0faf16b
clean sone callbacks
dvora-h Jun 1, 2023
5210e26
Merge branch '5.0' into resp3-response-callbacks
dvora-h Jun 1, 2023
b9698ef
first phase
chayim Jun 4, 2023
13b21c1
tox wrap back
chayim Jun 4, 2023
a24af93
Merge branch '5.0' into resp3-response-callbacks
dvora-h Jun 4, 2023
06dd87d
Merge branch '5.0' into ck-ciexp
chayim Jun 5, 2023
790da1a
changing cancel format
chayim Jun 5, 2023
e01e473
syntax
chayim Jun 6, 2023
b83db93
lint
chayim Jun 6, 2023
11c2c1b
docker
chayim Jun 6, 2023
7b37050
contain the docker
chayim Jun 6, 2023
4456d24
tox dev reqs
chayim Jun 7, 2023
defa742
back to testing
chayim Jun 7, 2023
c08d0ac
response callbacks
dvora-h Jun 14, 2023
1c7d3c0
Merge branch '5.0' into resp3-response-callbacks
chayim Jun 15, 2023
7ee1014
fixing sync tests post merge
chayim Jun 15, 2023
46c823a
protocol into async conftest
chayim Jun 15, 2023
3a5904a
fix for 3.11 invoke
chayim Jun 15, 2023
a7cb1b2
merged
chayim Jun 18, 2023
19257f8
docker changes
chayim Jun 18, 2023
1347342
fix tests
dvora-h Jun 18, 2023
7fb4ea2
linters
dvora-h Jun 18, 2023
3ff10ce
adding
chayim Jun 19, 2023
45869bd
resp3 tox, until killed
chayim Jun 19, 2023
85f4644
remove tox
chayim Jun 19, 2023
6a37856
tests
chayim Jun 19, 2023
b32c12e
requirements.txt
chayim Jun 19, 2023
75bbccd
restoring requirements.txt
chayim Jun 19, 2023
75f1a8f
adding a sleep, hopefully enough time for the cluster dockers to settle
chayim Jun 19, 2023
4d9bb58
fix search tests
dvora-h Jun 21, 2023
92b1fed
Merge branch 'ck-ciexp' of https://github.com/redis/redis-py into ck-…
dvora-h Jun 21, 2023
9a336fb
search test, disable uvloop for pypy due to bug
chayim Jun 25, 2023
fe6bf20
syn
chayim Jun 25, 2023
21f1709
reg
chayim Jun 25, 2023
3a6590a
dialect test improvement
chayim Jun 25, 2023
ef46761
sleep+, xfail
chayim Jun 25, 2023
b2776ff
tests
chayim Jun 25, 2023
8fba13d
resp
chayim Jun 25, 2023
5da235a
flaky search test too
chayim Jun 25, 2023
86cb137
timing
chayim Jun 25, 2023
9a8e7a6
timing for async test
chayim Jun 26, 2023
55dd6d4
test changes
chayim Jun 27, 2023
8e430e4
fix assert_interval_advanced
dvora-h Jun 27, 2023
bab62be
revert
dvora-h Jun 27, 2023
3010d6c
mark async health_check tests with xfail
dvora-h Jun 27, 2023
77d0aa2
change strict to false
dvora-h Jun 27, 2023
bf10aa5
fix github actions package validation
dvora-h Jun 27, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions .flake8
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
[flake8]
max-line-length = 88
exclude =
*.egg-info,
*.pyc,
.git,
.tox,
.venv*,
build,
docs/*,
dist,
docker,
venv*,
.venv*,
whitelist.py,
tasks.py
ignore =
F405
W503
E203
E126
75 changes: 62 additions & 13 deletions .github/workflows/integration.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,10 @@ on:
schedule:
- cron: '0 1 * * *' # nightly build

concurrency:
group: ${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

permissions:
contents: read # to fetch code (actions/checkout)

Expand Down Expand Up @@ -48,7 +52,7 @@ jobs:

run-tests:
runs-on: ubuntu-latest
timeout-minutes: 30
timeout-minutes: 60
strategy:
max-parallel: 15
fail-fast: false
Expand All @@ -68,32 +72,77 @@ jobs:
- name: run tests
run: |
pip install -U setuptools wheel
pip install -r requirements.txt
pip install -r dev_requirements.txt
tox -e ${{matrix.test-type}}-${{matrix.connection-type}}
if [ "${{matrix.connection-type}}" == "hiredis" ]; then
pip install hiredis
fi
invoke devenv
sleep 5 # time to settle
invoke ${{matrix.test-type}}-tests

- uses: actions/upload-artifact@v2
if: success() || failure()
with:
name: pytest-results-${{matrix.test-type}}
name: pytest-results-${{matrix.test-type}}-${{matrix.connection-type}}-${{matrix.python-version}}
path: '${{matrix.test-type}}*results.xml'

- name: Upload codecov coverage
uses: codecov/codecov-action@v3
if: ${{matrix.python-version == '3.11'}}
with:
fail_ci_if_error: false
# - name: View Test Results
# uses: dorny/test-reporter@v1
# if: success() || failure()
# with:
# name: Test Results ${{matrix.python-version}} ${{matrix.test-type}}-${{matrix.connection-type}}
# path: '${{matrix.test-type}}*results.xml'
# reporter: java-junit
# list-suites: failed
# list-tests: failed
# max-annotations: 10

- name: View Test Results
uses: dorny/test-reporter@v1
if: success() || failure()
continue-on-error: true
with:
name: Test Results ${{matrix.python-version}} ${{matrix.test-type}}-${{matrix.connection-type}}
path: '*.xml'
reporter: java-junit
list-suites: all
list-tests: all
max-annotations: 10
fail-on-error: 'false'

resp3_tests:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ['3.7', '3.11']
test-type: ['standalone', 'cluster']
connection-type: ['hiredis', 'plain']
protocol: ['3']
env:
ACTIONS_ALLOW_UNSECURE_COMMANDS: true
name: RESP3 [${{ matrix.python-version }} ${{matrix.test-type}}-${{matrix.connection-type}}]
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
- name: run tests
run: |
pip install -U setuptools wheel
pip install -r requirements.txt
pip install -r dev_requirements.txt
if [ "${{matrix.connection-type}}" == "hiredis" ]; then
pip install hiredis
fi
invoke devenv
sleep 5 # time to settle
invoke ${{matrix.test-type}}-tests
invoke ${{matrix.test-type}}-tests --uvloop

build_and_test_package:
name: Validate building and installing the package
runs-on: ubuntu-latest
needs: [run-tests]
strategy:
fail-fast: false
matrix:
extension: ['tar.gz', 'whl']
steps:
Expand Down
5 changes: 5 additions & 0 deletions .isort.cfg
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
[settings]
profile=black
multi_line_output=3
src_paths = ["redis", "tests"]
skip_glob=benchmarks/*
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,9 @@ Here's how to get started with your code contribution:
a. python -m venv .venv
b. source .venv/bin/activate
c. pip install -r dev_requirements.txt
c. pip install -r requirements.txt

4. If you need a development environment, run `invoke devenv`
4. If you need a development environment, run `invoke devenv`. Note: this relies on docker-compose to build environments, and assumes that you have a version supporting [docker profiles](https://docs.docker.com/compose/profiles/).
5. While developing, make sure the tests pass by running `invoke tests`
6. If you like the change and think the project could use it, send a
pull request
Expand All @@ -59,7 +60,6 @@ can execute docker and its various commands.
- Three sentinel Redis nodes
- A redis cluster
- An stunnel docker, fronting the master Redis node
- A Redis node, running unstable - the latest redis

The replica node, is a replica of the master node, using the
[leader-follower replication](https://redis.io/topics/replication)
Expand Down
5 changes: 2 additions & 3 deletions dev_requirements.txt
Original file line number Diff line number Diff line change
@@ -1,15 +1,14 @@
click==8.0.4
black==22.3.0
flake8==5.0.4
flake8-isort==6.0.0
flynt~=0.69.0
isort==5.10.1
mock==4.0.3
packaging>=20.4
pytest==7.2.0
pytest-timeout==2.0.1
pytest-timeout==2.1.0
pytest-asyncio>=0.20.2
tox==3.27.1
tox-docker==3.1.0
invoke==1.7.3
pytest-cov>=4.0.0
vulture>=2.3.0
Expand Down
109 changes: 109 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
---

version: "3.8"

services:

redis:
image: redis/redis-stack-server:edge
container_name: redis-standalone
ports:
- 6379:6379
environment:
- "REDIS_ARGS=--enable-debug-command yes --enable-module-command yes"
profiles:
- standalone
- sentinel
- replica
- all

replica:
image: redis/redis-stack-server:edge
container_name: redis-replica
depends_on:
- redis
environment:
- "REDIS_ARGS=--replicaof redis 6379"
ports:
- 6380:6379
profiles:
- replica
- all

cluster:
container_name: redis-cluster
build:
context: .
dockerfile: dockers/Dockerfile.cluster
ports:
- 16379:16379
- 16380:16380
- 16381:16381
- 16382:16382
- 16383:16383
- 16384:16384
volumes:
- "./dockers/cluster.redis.conf:/redis.conf:ro"
dvora-h marked this conversation as resolved.
Show resolved Hide resolved
profiles:
- cluster
- all

stunnel:
image: redisfab/stunnel:latest
depends_on:
- redis
ports:
- 6666:6666
profiles:
- all
- standalone
- ssl
volumes:
- "./dockers/stunnel/conf:/etc/stunnel/conf.d:ro"
- "./dockers/stunnel/keys:/etc/stunnel/keys:ro"

sentinel:
image: redis/redis-stack-server:edge
container_name: redis-sentinel
depends_on:
- redis
environment:
- "REDIS_ARGS=--port 26379"
entrypoint: "/opt/redis-stack/bin/redis-sentinel /redis.conf --port 26379"
ports:
- 26379:26379
volumes:
- "./dockers/sentinel.conf:/redis.conf"
profiles:
- sentinel
- all

sentinel2:
image: redis/redis-stack-server:edge
container_name: redis-sentinel2
depends_on:
- redis
environment:
- "REDIS_ARGS=--port 26380"
entrypoint: "/opt/redis-stack/bin/redis-sentinel /redis.conf --port 26380"
ports:
- 26380:26380
volumes:
- "./dockers/sentinel.conf:/redis.conf"
profiles:
- sentinel
- all

sentinel3:
image: redis/redis-stack-server:edge
container_name: redis-sentinel3
depends_on:
- redis
entrypoint: "/opt/redis-stack/bin/redis-sentinel /redis.conf --port 26381"
ports:
- 26381:26381
volumes:
- "./dockers/sentinel.conf:/redis.conf"
profiles:
- sentinel
- all
4 changes: 0 additions & 4 deletions docker/base/Dockerfile

This file was deleted.

11 changes: 0 additions & 11 deletions docker/base/Dockerfile.cluster

This file was deleted.

9 changes: 0 additions & 9 deletions docker/base/Dockerfile.cluster4

This file was deleted.

9 changes: 0 additions & 9 deletions docker/base/Dockerfile.cluster5

This file was deleted.

4 changes: 0 additions & 4 deletions docker/base/Dockerfile.redis4

This file was deleted.

4 changes: 0 additions & 4 deletions docker/base/Dockerfile.redis5

This file was deleted.

12 changes: 0 additions & 12 deletions docker/base/Dockerfile.redismod_cluster

This file was deleted.

4 changes: 0 additions & 4 deletions docker/base/Dockerfile.sentinel

This file was deleted.

4 changes: 0 additions & 4 deletions docker/base/Dockerfile.sentinel4

This file was deleted.

4 changes: 0 additions & 4 deletions docker/base/Dockerfile.sentinel5

This file was deleted.

11 changes: 0 additions & 11 deletions docker/base/Dockerfile.stunnel

This file was deleted.

18 changes: 0 additions & 18 deletions docker/base/Dockerfile.unstable

This file was deleted.

Loading