-
-
Notifications
You must be signed in to change notification settings - Fork 4.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
4.2.x high RAM memory usage due to HTTPS tracker announces for some users #12326
Comments
Cache settings were set to auto in 4.2.2. Maybe that could’ve caused this issue. Try setting it manually. |
Assuming all of the 2 GiB extra usage is cache, you'd have to have 80 GiB or more of system memory for libtorrent to allocate that much when the cache setting is set to auto. If you do have 80 GiB or more of system memory, then everything is working fine and the auto setting is just not suitable for your use case, in which case you should manually set it to some value that suits you. Otherwise, there is a bug in libtorrent (again, assuming all extra 2 GiB memory usage is cache). |
seems ok here.... 3 large torrents downloading..... @p43b1 can you post a screenshot of task manager > performance -> memory. |
To really get to the bottom of this, and figure out if this is a problem with qBIttorrent or the system, it is necessary to use a profiler. I don't know how it would be on Windows, but it has to be something similar to the (callgrind+kcachegrind, massif) combo on Linux. |
@FranciscoPombal I'll have to look into that kind of thing....I'm just wondering if "compressed memory" on/off makes a difference, I have it enabled....hence why I asked for task manager screenshot.......currently can't go fiddling/testing with settings/test machines. EDIT: Also @p43b1 do you have "Enable OS cache" checked or unchecked in the advanced settings? |
@xavier2k6 The Qt wiki also has a wiki page that names the first 2 among many others: |
@FranciscoPombal I have debugdiag attached 24/7 with full page heaps enabled......yet mem seems ok?! I know there's a memory leak alright in Qt which will be fixed in Qt 5.14.2 |
I think I may be having the same issue. Running on Linux with qbittorrent-nox 4.2.2 (libtorrent 1.2.5). Around 60 torrents with 8 active. Been running for ~4 hours with just over 2 GiB usage. I do have 512MiB disk cache set, but the total usage still seems excessive. Is this an issue or expected behaviour? UPDATE: Just ran into OOM, restarted qbittorrent-nox and it's at 100MiB usage |
@FranciscoPombal did you get any further debug/info out of #11879? |
All three taken at the very same moment Edit: |
@xavier2k6 no, since I had the problems with the alert patches mentioned near the end, I did not give it another go. Perhaps I should try again. @p43b1 Can MTuner show symbol names? Without them it is difficult to see where exactly the most memory is being allocated. Perhaps you need to point it at qBittorrent's .pdb file? Also can it export data in a massif-compatible format (or any format at all)? If so, post the export here so that others can analyze. EDIT: I see at least the stack trace in the bottom right seems to have symbol names, but the useful bits are on the top of the stack (which would be in the bottom of the list, not visible in the screenshot). Please export a snapshot of the data when you are experiencing the increased memory usage so that others can attempt to analyze. If this leak/increased usage is also related to the crypto library functions, this can be closed as a duplicate of the other issue and we can regroup and refocus our efforts over there. |
@FranciscoPombal quick observation & thinking out loud..... @p43b1 has a set/custom disk cache 128MiB where as I have auto -> which follows libtorrents limits/algorithm (for amount of ram in the system) I only presume that when a set/custom disk cache is used these limits are no longer adhered to......but is the set/custom disk cache figure restricted to just that amount & if it is....maybe the restrictor could be the issue/leaker..... |
You should compile and build normally (assuming you are on Ubuntu, here is the guide: https://github.com/qbittorrent/qBittorrent/wiki/Compiling-qBittorrent-on-Debian-and-Ubuntu) and then run at least This comment shows other examples for |
@p43b1 you can try auto, but I seriously doubt that choosing a specific value is causing a bug. As far as I know, that libtorrent code hasn't changed in a while. As an additional note, the amount of RAM used when auto is selected is chosen according to the following logic: |
I managed to capture it, summary:
Some details:
Looks like thread creation might be leaking things? I killed it after reaching almost 3Gi RSS, while downloading a big torrent. I'm not sure how to read it, but it seems to point the finger at spawning threads? But I strace-d it and it doesn't seem to clone any while downloading intensively 🤷♂️ |
@farnoy do you mind uploading the full output file, in case you saved it? You can simply attach it to your post. |
Also, if possible, compile and use libtorrent with debug symbols so that we can see the stacktrace for the libtorrent portion.
|
I will recompile libtorrent with debugging, for now, this is the whole file. |
Ok, I'm building with autotools and a simple
I'll try to compile a version with debug symbols, but with full optimizations and no assertions. |
@FranciscoPombal Here's one with debug symbols massif_output.zip I'm not sure this is correct, I killed it around 700Mi of RSS, but massif's summary is showing 255Mi peak memory used? |
No problems on Linux with 2648 runnung tasks: |
@p43b1 can you please compile with debug symbols/configure MTTuner to use debug symbols, and post the exported data so that others can analyze freely in more detail? I realize this is not the easiest thing to do on Windows, it's fine if you don't do it. But just as an example, with @farnoy's output posted above, I can open it and look at it in detail in |
It sounds like some people define "memory leak" as any meory allocated and not freed. I don't think that's a reasonable definition. A singleton that's allocated exactly once and then never freed is not a leak. It's freed by the operating system on process shutdown. This does not smell like a leak to me. Has anyone seen multiple instances leak? or even better, leaking over time, where longer wait means more allocated objects. |
9 more instances over a 12 hour period. click to view stack trace
click to view stack trace
click to view stack trace
click to view stack trace
click to view stack trace
click to view stack trace
click to view stack trace
click to view stack trace
click to view stack trace
|
@xavier2k6 Thanks for this. Seems there are important symbols missing from the stacktraces. For example, what are the libtorrent functions that the leaks originate from? I can only see this information in one of the stacktraces (3rd one from the top in your latest comment), where it seems to be related to the DHT. In all the other ones, how can you tell the issue comes from tracker connections? |
tooks me 4 days to verify that the leak do happened without patch on my setup, |
This was also confirmed by another user in a related ticket #12825 (comment) |
Is there a theoretical estimation of how much memory qbittorrent should use under default settings? My current setup is using 2487M (Total buffer size: 511.0 MiB) for seeding 3565 torrents. Averaging 500kB for each torrent, does that consider as high memory usage? |
@FranciscoPombal |
Can't say for certain if this is a good/bad value per-torrent (at least it does not look unreasonable). If the total usage is stable and not blowing up like the reports of previous posts, then at least you're not affected by this leak. |
libtorrent 1.2.9 is released, did anyone reproduce this issue with the new version? |
@mayli same here, using 2.5gb of memory with Archlinux packages. BTW, still 1.2.8 for https://www.archlinux.org/packages/extra/x86_64/libtorrent-rasterbar/ |
I have not managed to reproduce the issue with recent versions of the dependencies (i.e: the memory leak seems to be solved). After about 2 hours of use, the memory cache (2048MB in my case) has filled up as expected while seeding around 600 torrents (capped at 600KB/s upload) and downloading 255 torrents (capped at 500KB/s download). Once the cache finishes filling up, it looks like memory usage is relatively stable: I will report back here if I notice a leak after a couple of days of use. |
Mind testing with the latest commits of qBittorrent/libtorrent and possibly the latest releases of other dependencies as well? Ideally I think we would all like to get to the bottom of this, but if it go fixed somehow (e.g. possibly even due to some fix in boost or OpenSSL), we can close this and move on. |
For anyone curious, this was the patch in question: arvidn/libtorrent#4934. This might have been it, after all. |
1.2.9 is in testing, but results in https://bugs.archlinux.org/task/67754 |
The python bindings have a problem of their own, but otherwise the C++11/C++14 thing is only a problem when all of the following are true:
See https://github.com/arvidn/libtorrent/pull/5010/files for an explanation of the latter 2 points. Keep in mind that while all of these points being true will result in the symbol look up error, doing the exact opposite for some or all of them will also lead to problems. For example, if you build libtorrent with C++14, your client should not define The easiest "winning combos" are:
or, if building libtorrent with C++11:
|
Yeah, I mentioned in the bug report that we need libtorrent built with the C++14 std. But we also need to support deluge with the python bindings... We currently build it using the autotools configure script. Our libtorrent maintainer is going to take a look at resolving this tomorrow, the simplest solution is if there is a patch or configure parameter to guarantee that both libtorrent and the python bindings are built with C++14 using autotools. |
@guillaumedsde I can confirm the new 1.2.9 libtorrent seems stabilized the RAM usage to normal level, as well as cpu usage. update: I am hitting segfault on high speed downloading about 90MB/s on the static musl build, the dmesg says
my docker will automatically restart the process, but it keeps crashing for last 4 hours/ |
@mayli are you using the build from here https://github.com/userdocs/qbittorrent-nox-static ? |
@FranciscoPombal Yep. |
@userdocs Can you reproduce this? #12326 (comment) |
The seg fault? no. Using my latest musl build - https://github.com/userdocs/qbittorrent-nox-static/blob/master/bin/musl-static/amd64/qbittorrent-nox on Debian 10 Stable x64 Downloading Ubuntu at my almost max potential I get 0 warning in the console or my log files.
Does not meant the fault is not occurring, i just can't reproduce it on my Debian server. |
@slrslr This excessive memory usage issue is believed to be fixed on recent builds that use libtorrent >= 1.2.8. It would be nice if you could confirm this as well: https://github.com/FranciscoPombal/qBittorrent/actions/runs/241080799 (these recent |
I have not know which of the files to download, so i have downloaded first one and it was qbt 4.3.0alpha1 (libtorrent 1.2.10.0). i think that you are right, i see no memory issue anymore so far. Seeing big improvements from previous versions (After 4 hours of runtime on W10):
After i suspend the computer and on awake it like 5-10 minutes utilized system drive and CPU thread (23% usage on quad core CPU), seems normal to me. Then it was back to normal as before sleep. Back to the topic of this issue, so far memory issue seems to be fixed as @FranciscoPombal said. |
Since there have been no more complaints and multiple people on different platforms have reported good results, I think we can consider this to be addressed. So far it is believed this was the root cause: #12326 (comment) Users facing problems should try builds like https://github.com/FranciscoPombal/qBittorrent/actions/runs/241080799 or more recent - i.e., recent qBittorrent Thanks to everyone for their help and contributions!
If building libtorrent with C++11 and when building with CMake, passing |
Please provide the following information
qBittorrent version and Operating System
4.2.2
Windows 10
If on linux, libtorrent-rasterbar and Qt version
(type here)
What is the problem
Excessively high RAM memory usage compared to 4.2.0
320MB after continuously running for two weeks compared to 2.5GB after running for 3 hours with the same of downloading/uploading torrents.
4.2.2 RAM USAGE

4.2.2 STATISTICS
4.2.0 RAM USAGE

4.2.0 STATISTICS

What is the expected behavior
(type here)
Steps to reproduce
(type here)
Extra info(if any)
(type here)
The text was updated successfully, but these errors were encountered: