Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve performance for Python 3.x #751

Merged
merged 4 commits into from
Jun 14, 2016
Merged

Conversation

cjsimpson
Copy link
Contributor

This issue attempts to improve performance under Python 3.x while either maintaining or slightly improving 2.x performance.

Some of the reasons Python 3.x is slow than 2.x:

  • Strings need to be encoded into bytes before sending to redis
  • Many of these strings are encoded over and over again (get, set, incr, etc..)

These redis commands are represented by the connection.Token object. To speed things up, I've implemented a cache of Token objects and modified the Token object to store its encoded value. This greatly reduces the number of times redis commands need to be encoded. It also reduces the number of times Token objects are created, although that has a smaller impact.

I also wrote a basic benchmark (basic_operations.py) modeled on redis-benchmark (although much more limited). It tests the performance of some of the basic commands.

usage: basic_operations.py [-h] [-n] [-P] [-s]

optional arguments:
  -h, --help  show this help message and exit
  -n         Total number of requests (default 100000)
  -P         Pipeline <numreq> requests. Default 1 (no pipeline).
  -s         Data size of SET/GET value in bytes (default 2)

Testing on my machine has shown a 10-20 % improvement for Python 3.5. It should make the biggest improvement to workloads where there are small and frequent requests to redis.

cjsimpson added 4 commits June 9, 2016 20:10
The Token class now contains a cache of tokens, and each token stores its encoded value.
In Python 3 this prevents encoding the Token commands (get, set, incr, etc...) repeatly.
There is also a smaller performance improvement by creating fewer objects.

A very basic benchmark script was also added.
@andymccurdy
Copy link
Contributor

This looks great. Thanks!

@andymccurdy andymccurdy merged commit 0531abe into redis:master Jun 14, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants