Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a performance testing tool #68

Merged
merged 1 commit into from
Jan 28, 2022
Merged

Conversation

adamncasey
Copy link
Contributor

@adamncasey adamncasey commented Jan 24, 2022

This PR adds a performance test tool useful for testing amqpprox performance.

An extract from the README.md included in the performance_tester directory:

amqpprox Performance Testing

amqpprox performance has been tested mainly in two ways:

  1. Total data throughput achieved by connected clients (MB/s)
  2. Total connection establishment throughput achieved (connections/s)

The performance tester in this folder helps with both of these by:

  1. It contains a dummy AMQP 0.9.1. server which walks the AMQP handshake and ignores all future frames except close.
  2. It can run parallel AMQP clients connecting to an amqpprox instance.
    1. Testing data throughput probably wants to send more, larger messages with fewer connections
    2. Testing connection throughput probably wants to send fewer, smaller messages with many connections.

Usage Help

amqpprox_perf_tester

USAGE:
    amqpprox_perf_tester [OPTIONS] --listen-address <LISTEN_ADDRESS>

OPTIONS:
        --address <ADDRESS>                  [default: amqp://localhost:5672/]
        --clients <CLIENTS>                  [default: 10]
    -h, --help                               Print help information
        --listen-address <LISTEN_ADDRESS>
        --listen-cert <LISTEN_CERT>
        --listen-key <LISTEN_KEY>
        --max-threads <MAX_THREADS>          [default: 50]
        --message-size <MESSAGE_SIZE>        [default: 100]
        --num-messages <NUM_MESSAGES>        [default: 10]

PR todo

  • Fix any inevitable CI issues - off the top of my head the image we run the integration tests in will need to build rust code.

tests/performance_tester/src/client.rs Outdated Show resolved Hide resolved
tests/performance_tester/src/main.rs Outdated Show resolved Hide resolved
tests/performance_tester/src/main.rs Outdated Show resolved Hide resolved
tests/performance_tester/README.md Outdated Show resolved Hide resolved
tests/performance_tester/integration-tests.py Outdated Show resolved Hide resolved
tests/performance_tester/src/main.rs Outdated Show resolved Hide resolved
This commit adds a new test tool to the amqpprox repo. Extract from the
new `tests/performance_tester/README.md`:

```

`amqpprox` performance has been tested mainly in two ways:

1. Total data throughput achieved by connected clients (MB/s)
2. Total connection establishment throughput achieved (connections/s)

The performance tester in this folder helps with both of these by:

1. It contains a dummy AMQP 0.9.1. server which walks the AMQP handshake and ignores all future frames except close.
2. It can run parallel AMQP clients connecting to an amqpprox instance.
    1. Testing data throughput probably wants to send more, larger messages with fewer connections
    2. Testing connection throughput probably wants to send fewer, smaller messages with many connections.
```

I tried lapin instead of amiquip when investigating adding TLS
support for the `client -> amqpprox` hop, but found lapin performed
significantly worse than `amiquip` and often failed to shutdown cleanly.

It's possible this is because amiquip was given a dedicated thread and lapin
ran async, or perhaps too many async clients were started & failed to make
progress. But initial numbers were 75% down. I didn't think this would properly
exercise amqpprox so dropped it for now. It's possible we just need to
go down the dummy AMQP client route too.

My test setup results so far indicate a roughly 50% impact in connection
throughput when enabling TLS from amqpprox to the broker: 2000+
connections/s down to around ~1000/s. Overall data
throughput is affected significantly less, with a ~5% reduction: 690MB/s
-> 630MB/s. This test setup runs amqpprox on a different, nearby,
machine to amqpprox_perf_tester, on similar-ish spec machines to what we
use in production. Although I've mainly been looking at the difference
in performance here, the absolute numbers are interesting too. I am not
sure how we end up 'only' achieving 700MB/s, it's possible the test
client can't fully utilise amqpprox here.

This commit also upgrades the integration tests to rmq 3.7.28 dockerhub image
since 3.7.9 hasn't been built in over 3 years. I had upgraded to 3.9 but that
didn't pass the integration tests. I'd prefer to investigate that separately

Co-authored-by: Alaric <alaric@bloomberg.net>
@adamncasey
Copy link
Contributor Author

Squash & merge is disabled on this repo, possibly because of CLA concerns. I squashed locally

@adamncasey adamncasey merged commit 6b87dfc into bloomberg:main Jan 28, 2022
@adamncasey adamncasey deleted the perftesttls branch January 28, 2022 11:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants