Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sa-east-1 (Sao Paulo) -- > elasticache eu-central-1 (Frankfurt) ~200ms 3 limits one 1 with low time periods #88

Closed
Tracked by #62
eguzki opened this issue Mar 22, 2024 · 0 comments
Closed
Tracked by #62

sa-east-1 (Sao Paulo) -- > elasticache eu-central-1 (Frankfurt) ~200ms 3 limits one 1 with low time periods #88

eguzki opened this issue Mar 22, 2024 · 0 comments

Comments

@eguzki
Copy link
Contributor

eguzki commented Mar 22, 2024 •
edited
Loading

Test setup

Elasticache <- EC2 Redis Proxy (socat)  < ---------------- INET --------------- EC2 Limitador - Driver

Redis proxy

1 EC2 instance

amazon linux 2023 AMI

type: c7i.large 2vCPU 4Gib Memory

sudo yum update -y
sudo yum install -y socat
socat TCP-LISTEN:9876,fork,reuseaddr TCP:${ELASTICACHE_PRIMARY_ENDPOINT}

Elasticache

  • Deployment option -> Design your own cache
  • Creation Method -> Cluster cache
  • Cluster mode -> disabled
  • Multi-AZ -> disabled
  • Auto-failover -> enabled
  • Engine version -> 7.0.7
  • Node type -> cache.m5.2xlarge (x86_64, 8vCPU, 26GiB memory, Enhanced I/O, Enhanced I/O Multiplexing)
  • VPC -> same as the client EC2 instance
  • AZ -> us-east-1a
  • 2 node (1 replicas)
  • Backup -> disabled
  • Security groups -> Allow TCP traffic to port 6379 only from client node.
  • Logs -> disabled

Limtador and Driver

1 EC2 instance

amazon linux 2023 AMI

type: c7i xlarge 4vCPU 8Gib Memory

sudo yum update -y
sudo yum install -y docker
sudo service docker start
sudo usermod -a -G docker ec2-user
# (logout and login again)
docker ps

Create docker image with limitador and limitador-driver

cat <<'EOF' >Dockerfile
ARG LIMITADOR_IMAGE

FROM $LIMITADOR_IMAGE as limitador

# Stage 1: Build limitador-driver
FROM rust:1.76-bullseye as builder

ARG LIMITADOR_DRIVER_REPO=https://github.com/alexsnaps/limitador-driver.git
ARG LIMITADOR_DRIVER_BRANCH=main

RUN apt update && apt upgrade -y \
    && apt install -y protobuf-compiler git

RUN git clone --depth 1 $LIMITADOR_DRIVER_REPO -b $LIMITADOR_DRIVER_BRANCH /opt/limitador-driver

WORKDIR /opt/limitador-driver

RUN cargo build --release

# Stage 2: Setup the environment to run limitador-driver and limitador
FROM rust:1.76-bullseye

# Copy the Rust binary from the builder stage
COPY --from=builder /opt/limitador-driver/target/release/limitador-driver /usr/local/bin

# Copy the limitador service into the container
COPY --from=limitador /home/limitador/bin/limitador-server /usr/local/bin

# Run the script
CMD "/bin/bash"
EOF

Build image based on some limitador image, for example quay.io/kuadrant/limitador:write-behind-lock

docker build --build-arg="LIMITADOR_IMAGE=quay.io/kuadrant/limitador:write-behind-lock" -t write-behind-lock .

Limits file

cat <<EOF >limits.yaml
---
- namespace: test_namespace
  max_value: 10000000
  seconds: 3600
  conditions:
  variables:
- namespace: test_namespace
  max_value: 10000000
  seconds: 1800
  conditions:
  variables:
- namespace: test_namespace
  max_value: 100000
  seconds: 5
  conditions:
  variables:
EOF

Start container

docker run --name limitador-driver --rm -ti -v $PWD/limits.yaml:/tmp/limits.yaml write-behind-lock

Start limitador

/usr/local/bin/limitador-server -vv /tmp/limits.yaml redis_cached --ratio 1 redis://$REDIS_PROXY:9876
Limitador Server v1.4.0-dev (83fe4e51) [] release
2024-03-22T16:13:35.469516Z  INFO limitador_server: Version: v1.4.0-dev (83fe4e51) [] release    
2024-03-22T16:13:35.469537Z  INFO limitador_server: Using config: Configuration { limits_file: "/tmp/limits.yaml", storage: Redis(RedisStorageConfiguration { url: "redis://ec2-54-93-63-180.eu-central-1.compute.amazonaws.com:9876", cache: Some(RedisStorageCacheConfiguration { flushing_period: 1000, max_ttl: 5000, ttl_ratio: 1, max_counters: 10000 }) }), rls_host: "0.0.0.0", rls_port: 8081, http_host: "0.0.0.0", http_port: 8080, limit_name_in_labels: false, tracing_endpoint: "", log_level: Some(LevelFilter::INFO), rate_limit_headers: None, grpc_reflection_service: false }    
2024-03-22T16:13:35.672703Z  INFO limitador_server: limits file path: /tmp/limits.yaml    
2024-03-22T16:13:35.673178Z  INFO limitador_server: Envoy RLS server starting on 0.0.0.0:8081    
2024-03-22T16:13:35.673190Z  INFO limitador_server: HTTP server starting on 0.0.0.0:8080    
2024-03-22T16:13:35.673396Z  INFO actix_server::builder: starting 4 workers
2024-03-22T16:13:35.673404Z  INFO actix_server::server: Actix runtime found; starting in Actix runtime

Open a new shell the ec2 instance

docker exec -ti limitador-driver /bin/bash

Run the driver to start the test

/usr/local/bin/limitador-driver rpc://127.0.0.1:8081 1m

Results

root@3c37cb816fd1:/# /usr/local/bin/limitador-driver rpc://127.0.0.1:8081 1m
Hitting Limitador at rpc://127.0.0.1:8081 for a total duration of 1 minutes
Current: hits: 9725, mean: 0.102ms, p50: 0.078ms, p90: 0.091ms, p99: 0.117ms, p999: 0.369ms, max: 201.727ms
Current: hits: 12392, mean: 0.080ms, p50: 0.077ms, p90: 0.089ms, p99: 0.112ms, p999: 0.277ms, max: 0.405ms
Current: hits: 12288, mean: 0.081ms, p50: 0.077ms, p90: 0.090ms, p99: 0.118ms, p999: 0.333ms, max: 0.555ms
Current: hits: 12304, mean: 0.081ms, p50: 0.077ms, p90: 0.090ms, p99: 0.113ms, p999: 0.293ms, max: 0.419ms
Current: hits: 12256, mean: 0.081ms, p50: 0.077ms, p90: 0.091ms, p99: 0.115ms, p999: 0.307ms, max: 0.575ms
Current: hits: 1708, mean: 0.585ms, p50: 0.077ms, p90: 0.092ms, p99: 0.353ms, p999: 323.583ms, max: 329.727ms
Current: hits: 12144, mean: 0.082ms, p50: 0.078ms, p90: 0.092ms, p99: 0.118ms, p999: 0.297ms, max: 0.403ms
Current: hits: 12195, mean: 0.081ms, p50: 0.078ms, p90: 0.091ms, p99: 0.117ms, p999: 0.291ms, max: 0.415ms
Current: hits: 12263, mean: 0.081ms, p50: 0.077ms, p90: 0.091ms, p99: 0.114ms, p999: 0.313ms, max: 0.453ms
Current: hits: 12127, mean: 0.082ms, p50: 0.078ms, p90: 0.092ms, p99: 0.117ms, p999: 0.297ms, max: 0.531ms
Current: hits: 1746, mean: 0.573ms, p50: 0.077ms, p90: 0.091ms, p99: 0.227ms, p999: 325.631ms, max: 329.727ms
Current: hits: 7309, mean: 0.180ms, p50: 0.077ms, p90: 0.090ms, p99: 0.118ms, p999: 0.305ms, max: 403.455ms
Current: hits: 12357, mean: 0.080ms, p50: 0.077ms, p90: 0.090ms, p99: 0.113ms, p999: 0.277ms, max: 0.647ms
Current: hits: 12199, mean: 0.081ms, p50: 0.078ms, p90: 0.091ms, p99: 0.116ms, p999: 0.299ms, max: 0.475ms
Current: hits: 8146, mean: 0.122ms, p50: 0.079ms, p90: 0.092ms, p99: 0.121ms, p999: 0.313ms, max: 323.583ms
Current: hits: 9742, mean: 0.102ms, p50: 0.078ms, p90: 0.091ms, p99: 0.117ms, p999: 0.315ms, max: 201.727ms
Current: hits: 7292, mean: 0.136ms, p50: 0.078ms, p90: 0.091ms, p99: 0.117ms, p999: 0.339ms, max: 401.407ms
Current: hits: 2432, mean: 0.411ms, p50: 0.078ms, p90: 0.092ms, p99: 0.168ms, p999: 0.555ms, max: 403.455ms
Current: hits: 12160, mean: 0.082ms, p50: 0.078ms, p90: 0.091ms, p99: 0.119ms, p999: 0.299ms, max: 0.435ms
Current: hits: 8411, mean: 0.142ms, p50: 0.077ms, p90: 0.089ms, p99: 0.113ms, p999: 0.277ms, max: 323.583ms
Current: hits: 12216, mean: 0.081ms, p50: 0.078ms, p90: 0.091ms, p99: 0.115ms, p999: 0.287ms, max: 0.419ms
Current: hits: 12256, mean: 0.081ms, p50: 0.078ms, p90: 0.091ms, p99: 0.114ms, p999: 0.277ms, max: 0.355ms
Current: hits: 2362, mean: 0.423ms, p50: 0.078ms, p90: 0.093ms, p99: 0.139ms, p999: 0.583ms, max: 403.455ms
Current: hits: 12191, mean: 0.081ms, p50: 0.078ms, p90: 0.091ms, p99: 0.114ms, p999: 0.303ms, max: 0.473ms
Current: hits: 5710, mean: 0.174ms, p50: 0.078ms, p90: 0.092ms, p99: 0.133ms, p999: 0.443ms, max: 323.583ms
Current: hits: 12025, mean: 0.082ms, p50: 0.079ms, p90: 0.092ms, p99: 0.119ms, p999: 0.311ms, max: 0.603ms
Current: hits: 12234, mean: 0.081ms, p50: 0.078ms, p90: 0.091ms, p99: 0.118ms, p999: 0.275ms, max: 0.329ms
Current: hits: 9756, mean: 0.102ms, p50: 0.077ms, p90: 0.091ms, p99: 0.117ms, p999: 0.329ms, max: 202.751ms
Current: hits: 7256, mean: 0.137ms, p50: 0.078ms, p90: 0.092ms, p99: 0.123ms, p999: 0.357ms, max: 399.359ms
Current: hits: 5695, mean: 0.175ms, p50: 0.079ms, p90: 0.093ms, p99: 0.124ms, p999: 0.401ms, max: 323.583ms
Current: hits: 12129, mean: 0.082ms, p50: 0.078ms, p90: 0.092ms, p99: 0.117ms, p999: 0.303ms, max: 0.711ms
Current: hits: 12245, mean: 0.081ms, p50: 0.077ms, p90: 0.091ms, p99: 0.116ms, p999: 0.287ms, max: 0.403ms
Current: hits: 12370, mean: 0.080ms, p50: 0.077ms, p90: 0.090ms, p99: 0.112ms, p999: 0.255ms, max: 0.315ms
Current: hits: 7314, mean: 0.136ms, p50: 0.078ms, p90: 0.091ms, p99: 0.125ms, p999: 0.467ms, max: 397.311ms
Current: hits: 2385, mean: 0.586ms, p50: 0.078ms, p90: 0.092ms, p99: 0.142ms, p999: 397.311ms, max: 403.455ms
Current: hits: 12251, mean: 0.081ms, p50: 0.077ms, p90: 0.090ms, p99: 0.116ms, p999: 0.327ms, max: 0.469ms
Current: hits: 12207, mean: 0.081ms, p50: 0.077ms, p90: 0.091ms, p99: 0.119ms, p999: 0.293ms, max: 0.509ms
Current: hits: 12196, mean: 0.081ms, p50: 0.077ms, p90: 0.091ms, p99: 0.118ms, p999: 0.315ms, max: 0.551ms
Current: hits: 12258, mean: 0.113ms, p50: 0.077ms, p90: 0.090ms, p99: 0.116ms, p999: 0.279ms, max: 403.455ms
Current: hits: 4768, mean: 0.209ms, p50: 0.078ms, p90: 0.093ms, p99: 0.126ms, p999: 0.333ms, max: 403.455ms
Current: hits: 12160, mean: 0.082ms, p50: 0.078ms, p90: 0.092ms, p99: 0.117ms, p999: 0.301ms, max: 0.487ms
Current: hits: 12203, mean: 0.081ms, p50: 0.077ms, p90: 0.091ms, p99: 0.118ms, p999: 0.279ms, max: 0.403ms
Current: hits: 12197, mean: 0.081ms, p50: 0.078ms, p90: 0.091ms, p99: 0.117ms, p999: 0.289ms, max: 0.367ms
Current: hits: 12032, mean: 0.082ms, p50: 0.078ms, p90: 0.092ms, p99: 0.123ms, p999: 0.367ms, max: 0.571ms
Current: hits: 4823, mean: 0.207ms, p50: 0.077ms, p90: 0.090ms, p99: 0.127ms, p999: 0.427ms, max: 403.455ms
Current: hits: 12262, mean: 0.081ms, p50: 0.077ms, p90: 0.091ms, p99: 0.114ms, p999: 0.289ms, max: 0.401ms
Current: hits: 12102, mean: 0.082ms, p50: 0.078ms, p90: 0.092ms, p99: 0.120ms, p999: 0.289ms, max: 0.353ms
Current: hits: 12113, mean: 0.082ms, p50: 0.078ms, p90: 0.092ms, p99: 0.124ms, p999: 0.363ms, max: 0.687ms
Current: hits: 12096, mean: 0.082ms, p50: 0.078ms, p90: 0.091ms, p99: 0.127ms, p999: 0.385ms, max: 0.699ms
Current: hits: 4, mean: 251.565ms, p50: 201.727ms, p90: 403.455ms, p99: 403.455ms, p999: 403.455ms, max: 403.455ms
Current: hits: 12120, mean: 0.082ms, p50: 0.078ms, p90: 0.092ms, p99: 0.124ms, p999: 0.343ms, max: 0.627ms
Current: hits: 12180, mean: 0.081ms, p50: 0.078ms, p90: 0.091ms, p99: 0.121ms, p999: 0.291ms, max: 0.375ms
Current: hits: 12238, mean: 0.081ms, p50: 0.077ms, p90: 0.091ms, p99: 0.119ms, p999: 0.289ms, max: 0.559ms
Current: hits: 12239, mean: 0.097ms, p50: 0.077ms, p90: 0.090ms, p99: 0.114ms, p999: 0.315ms, max: 201.727ms
Current: hits: 6, mean: 201.016ms, p50: 0.389ms, p90: 403.455ms, p99: 403.455ms, p999: 403.455ms, max: 403.455ms
Current: hits: 12246, mean: 0.081ms, p50: 0.078ms, p90: 0.091ms, p99: 0.114ms, p999: 0.297ms, max: 0.409ms
Current: hits: 12046, mean: 0.082ms, p50: 0.078ms, p90: 0.092ms, p99: 0.124ms, p999: 0.323ms, max: 0.547ms
Current: hits: 12041, mean: 0.082ms, p50: 0.078ms, p90: 0.092ms, p99: 0.122ms, p999: 0.309ms, max: 0.507ms


 ============== Done! =================

Ok: hits: 573377, mean: 0.104ms, p50: 0.078ms, p90: 0.091ms, p99: 0.118ms, p999: 0.315ms, max: 403.455ms
OverLimit: hits: 0, mean: 0.000ms, p50: 0.000ms, p90: 0.000ms, p99: 0.000ms, p999: 0.000ms, max: 0.000ms
Unknown: hits: 0, mean: 0.000ms, p50: 0.000ms, p90: 0.000ms, p99: 0.000ms, p999: 0.000ms, max: 0.000ms
Overall: hits: 573377, mean: 0.104ms, p50: 0.078ms, p90: 0.091ms, p99: 0.118ms, p999: 0.315ms, max: 403.455ms

0.01ms |                                                                                  |  0.0th %-ile
0.02ms |                                                                                  |  0.0th %-ile
0.03ms |                                                                                  |  0.0th %-ile
0.04ms |                                                                                  |  0.0th %-ile
0.06ms | *                                                                                |  0.0th %-ile
0.07ms | *                                                                                |  0.8th %-ile
0.08ms | *********************************                                                | 41.2th %-ile
0.09ms | *********************************                                                | 81.8th %-ile
0.10ms | ************                                                                     | 96.1th %-ile
0.11ms | **                                                                               | 98.4th %-ile
0.12ms | *                                                                                | 99.1th %-ile
0.13ms | *                                                                                | 99.4th %-ile
0.14ms | *                                                                                | 99.5th %-ile
0.15ms | *                                                                                | 99.6th %-ile
0.17ms | *                                                                                | 99.6th %-ile
0.18ms | *                                                                                | 99.6th %-ile
0.19ms | *                                                                                | 99.7th %-ile
0.20ms | *                                                                                | 99.7th %-ile
0.21ms | *                                                                                | 99.7th %-ile
0.22ms | *                                                                                | 99.7th %-ile
0.23ms | *                                                                                | 99.7th %-ile
0.24ms | *                                                                                | 99.7th %-ile
0.25ms | *                                                                                | 99.7th %-ile
0.26ms | *                                                                                | 99.8th %-ile
0.28ms | *                                                                                | 99.8th %-ile
0.29ms | *                                                                                | 99.8th %-ile
0.30ms | *                                                                                | 99.9th %-ile
0.31ms | *                                                                                | 99.9th %-ile
0.32ms | *                                                                                | 99.9th %-ile
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@eguzki