Per-call memory growth with derive_private_key

[bfxtsales]

Would unbounded memory growth with repeated calls to derive_private_key be expected behavior, or am I doing something wrong or unsupported? Behavior is the same across Ubuntu and Mac and multiple python versions, all with Cryptography 2.1.4, as noted below.

This snippet experiences memory growth with each call to derive_private_key (with or without the "del sk" line). I tried some python profilers but can't track it down.

from cryptography.hazmat.primitives.asymmetric.ec import SECP256K1, derive_private_key
from cryptography.hazmat.backends import default_backend
DEFAULT_BACKEND = default_backend()
SECP256K1 = SECP256K1()
for i in range(0,10000000):
	sk = derive_private_key(1, SECP256K1, DEFAULT_BACKEND)
	del sk

Platforms
Linux ubuntu 4.13.0-32-generic #35-Ubuntu SMP Thu Jan 25 09:13:46 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Python 2.7.14 (default, Sep 23 2017, 22:06:14) [GCC 7.2.0] on linux2
Python 3.6.3 (default, Oct 3 2017, 21:45:48) [GCC 7.2.0] on linux
Cryptography 2.1.4
OpenSSL 1.0.2g 1 Mar 2016

macOS 10.12.6 (16G1114) Darwin 16.7.0
Python 2.7.10 (default, Feb 7 2017, 00:08:15) [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)] on darwin
Python 3.6.3 (default, Oct 4 2017, 06:09:15) [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.37)] on darwin
Cryptography 2.1.4
OpenSSL 1.0.2n 7 Dec 2017

Thank you.

[reaperhulk]

Thank you for the report, this is a memory leak. When calling EC_KEY_set_private_key the code erroneously assumed the BN * holding the private scalar had ownership taken by the EC_KEY, but in reality it is duped. This results in a memory leak of a BN * holding the private scalar on every invocation of derive_private_key.

[bfxtsales]

Thank you Paul. By the way, how do I update my venv's given the new code on github? I've tried with pip but I end up with the 2.1.4 version from November. I'm quite new at this :)

On Sun, Feb 4, 2018 at 5:30 AM, Paul Kehrer ***@***.***> wrote: Thank you for the report, this is a memory leak. When calling EC_KEY_set_private_key the code erroneously assumed the BN * holding the private scalar had ownership taken by the EC_KEY, but in reality it is duped. This results in a memory leak of a BN * holding the private scalar on every invocation of derive_private_key. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#4095 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AidEscYxa1iuN_EoWCC8LcEfA9cgK42Xks5tRbD7gaJpZM4R4iXr> .

<http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> Virus-free. www.avg.com <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

[reaperhulk]

@bfxtsales Until we do a release you'll need to clone this repo and install from source. You can do that with git clone https://github.com/pyca/cryptography && cd cryptography && pip install . on Linux. On macOS you'll need to also supply CFLAGS/LDFLAGS containing the paths to the OpenSSL you want to compile against (see: https://cryptography.io/en/latest/installation/)