Cawbird consumes a lot of memory over time

[IBBoard]

Describe the bug
It's not noticeable (at least not on my machine) but if you leave Cawbird running then it'll slowly consume more and more memory.

This isn't just to do with new tweets stacking up when it isn't auto-scrolling, because scrolling to the top and waiting for it to clear the older tweets doesn't reduce the memory usage by much.

To Reproduce
Steps to reproduce the behavior:

1. Open Cawbird and set it so that won't scroll with new tweets
2. Leave it running for a few hours so that 100+ new tweets appear
3. Check memory usage with ps aux | grep "[c]awbird" (column 4 is system RAM percentage, 5 is absolute amount)
4. Scroll to the top
5. Check the memory usage again

Expected behavior
At step 3, Cawbird is using some extra memory compared to startup, but not much.

At step 5, it's back down to startup levels.

** Actual behaviour **
At step 3:
ibboard 4665 0.6 4.1 1612168 1015320 ? Sl 10:36 1:33 /usr/bin/cawbird --gapplication-service
At step 5:
ibboard 4665 0.7 3.9 1505916 965580 ? Sl 10:36 2:02 /usr/bin/cawbird --gapplication-service
(Values after ~5hrs running)

System details:

• OS: openSUSE Tumbleweed
• Version: 1.0.5
• Package type: official RPM

Additional context
If anyone knows about understanding memory usage of an app (beyond running a basic Valgrind and seeing if it reports memory leaks) then please help!

[IBBoard]

Possible candidate cause (but I'm just starting to learn C/memory analysis tools):

90.71% (92,283,097B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->44.25% (45,017,836B) 0x6353639: ??? (in /usr/lib64/libpixman-1.so.0.36.0)
| ->44.25% (45,017,836B) 0x63536D8: ??? (in /usr/lib64/libpixman-1.so.0.36.0)
|   ->44.25% (45,017,836B) 0x534FD6E: ??? (in /usr/lib64/libcairo.so.2.11600.0)
|     ->44.24% (45,007,364B) 0x1A3894: load_animation (CbMediaDownloader.c:110)
|     | ->44.24% (45,007,364B) 0x1A4354: cb_media_downloader_load_threaded (CbMediaDownloader.c:376)
⋮

[schmittlauch]

I noticed this as well. Does cawbird delete posts (and more important: their media attachments) from memory overtime and during scrolling?
My behaviour is that I often scroll quite far to the bottom to read the last hours of timeline, then jump back to the top and then manually scroll upwards when new content appears.

[IBBoard]

It should delete them, and from a quick bit of debugging then I can see that it is disposing of media when it removes tweets (so scrolling back would use lots of memory as it loads tweets and images, but get to the top and it'll purge the old ones and free the memory). The media for the thumbnails does appear to be coming out at up to 1200×1200, though, so scroll back a few hours past a load of pictures and those thumbnails could quickly consume quite a bit of memory.

This probably wouldn't be so much of a problem if we weren't doing stuff in C! I'm just starting on some debugging code to understand what it's doing.

[IBBoard]

Trying to work out whether we are actually using a lot of memory, or whether VSZ is just a misleading number and the OS would reclaim a load if it needed to.

Progression of ps aux | grep [c]awbird

ibboard  20281  0.3  1.3 1108224 342884 pts/0  Sl+  12:10   0:43 build/cawbird
…
ibboard  20281  0.2  2.4 1196256 611972 pts/0  Rl+  12:10   1:17 build/cawbird
…
ibboard  20281  0.3  2.3 1241736 577716 pts/0  Sl+  12:10   1:32 build/cawbird
…
ibboard  20281  0.3  2.3 1241736 577716 pts/0  Sl+  12:10   1:34 build/cawbird
…
ibboard  20281  0.4  2.2 2116432 540372 pts/0  Sl+  12:10   2:15 build/cawbird

Given that we're potentially (in my case) loading dozens of 1200x1200px images and holding that it the Cb.Media object before scaling dynamically to ~400x400px then that may explain the 500MB resident (RSS) because each one is 32MB as a bitmap and I don't know what optimisation (if any) GdkPixmap might do. The name "pixmap" suggests none.

For reference, free -m is showing:

              total        used        free      shared  buff/cache   available
Mem:          23982        5177       11158         432        7646       18040
Swap:          5113           0        5113

So I'm nowhere near needing the OS to collect any free'd memory.

[IBBoard]

Yeah, I think we might be freeing memory, and we just need to do something more sensible with images.

Steps:

• Load Cawbird
• Run ps aux | grep [c]awbird
  • VSZ and RSS: 1092628 367024
• Scroll down one tweet
• Scroll up one tweet
• Cawbird prunes old tweets
• Run ps aux | grep [c]awbird
  • VSZ and RSS: 1092628 200632

So it has pruned ~160MB of RSS, but VSZ has remained the same. Which suggests it was free'ing the media correctly, but VSZ doesn't instantly come down?

[IBBoard]

Interestingly, I just came across this article, which has a different command to the basic process listing that I'd been using above:

ps -eo 'pid user rss:8 size:8 cmd' --sort 'rss'

Comparing the two, I get:
ibboard 2890 3.7 1.9 1282952 479896 ? Sl 18:50 1:22 /usr/bin/cawbird --gapplication-service
2890 ibboard 479896 496716 /usr/bin/cawbird --gapplication-service

So the virtual size is much closer to the resident size. man ps says that size (rather than vsz) is "approximate amount of swap space that would be required if the process were to dirty all writable pages and then be swapped out", which does feel like "actual memory usage" (i.e. vsz includes loads of spare, unused space).

Although this other article suggests that VSZ is everything that has been allocated, not just looked at. Which leaves me wondering if there's something we should be doing to make it get un-allocated.

I hate low-level languages and memory management!

[IBBoard]

Just ran /usr/bin/time -f "RSS: %MkiB" cawbird, which the Daniel Lange article in the previous post (first link) said will show maximum RSS when I quit. I only ran for five minutes or so, but it only gave RSS: 280056kiB.

I think this is to do with thumbnails an unnecessary scaling rather than any kind of memory leaks.

So, the solution is simple. Stop showing images!

Or I could do something better with the thumbnails and caching and see if that helps. A quick hack that I did the other night affected the displayed image size as well.

[bbhtt]

So, the solution is simple. Stop showing images!

Does it help? It shoots up to ~>500 Mb in just 3 srolls and I'm hiding images in my timeline. Scrolling and loading new tweets is the trigger. Will running Valgrind help?

[IBBoard]

I've had a quick look at the code and it looks like it downloads images and stores them as objects in memory regardless of whether you have images visible in the timeline or not. That's good for responsiveness and consistency (if you turn them on then you won't be waiting for them to be downloaded) but not for memory usage.

The suggestion of stopping showing images was a non-serious solution of fixing memory usage by just not showing images at all.

What metric are you using for ">500MB"? There's a couple of options in the previous comments that very rarely get to that level for me, even after long runs. And my first comment after the initial report is valgrind output.

[bbhtt]

What metric are you using for ">500MB"? There's a couple of options in the previous comments that very rarely get to that level for me, even after long runs.

I ran memory profiler and plotter, here's a graph, I made 3 full scrolls,opened 2 tweets one with picture one without,photos were hidden in timeline. If I leave it like this it goes around 570-600 Mb in task manager which is more than my browser with two tabs open. Is this normal? I missed that output.

[IBBoard]

But which version of "memory usage" is that using? RSS (resident set size - current actual memory usage) or VSZ (virtual size - effectively the potential memory usage, including stuff it's not using). All of which is made to look worse by the fact that those numbers may include the size of all of the shared libraries (gtk, etc), which then get double-counted across the system.

At least that's what I understand from a lot of reading around. Memory usage isn't a simple "this app is using this amount of my memory" thing.

As for what's "normal", whatever it does now is normal for the way it is written. Valgrind isn't showing any memory leaks, so we're using exactly what GTK/libsoup/various other libraries need to use. So the only way to get better (lower) is to improve what we hold in memory.

[IBBoard]

On the "shared libraries" front, Firefox links the following:

/lib64/ld-linux-x86-64.so.2
libc.so.6
libdl.so.2
libgcc_s.so.1
libm.so.6
libpthread.so.0
libstdc++.so.6
linux-vdso.so.1

In contrast, Cawbird links:

/lib64/ld-linux-x86-64.so.2
libatk-1.0.so.0
libatk-bridge-2.0.so.0
libatspi.so.0
libblkid.so.1
libbrotlicommon.so.1
libbrotlidec.so.1
libbz2.so.1
libcairo-gobject.so.2
libcairo.so.2
libcom_err.so.2
libcrypto.so.1.1
libc.so.6
libdatrie.so.1
libdbus-1.so.3
libdl.so.2
libdw.so.1
libEGL.so.1
libelf.so.1
libenchant-2.so.2
libepoxy.so.0
libexpat.so.1
libffi.so.8
libfontconfig.so.1
libfreetype.so.6
libfribidi.so.0
libgcrypt.so.20
libgdk-3.so.0
libgdk_pixbuf-2.0.so.0
libgio-2.0.so.0
libGLdispatch.so.0
libglib-2.0.so.0
libGL.so.1
libGLX.so.0
libgmodule-2.0.so.0
libgobject-2.0.so.0
libgpg-error.so.0
libgraphite2.so.3
libgspell-1.so.2
libgssapi_krb5.so.2
libgstreamer-1.0.so.0
libgtk-3.so.0
libharfbuzz.so.0
libidn2.so.0
libjson-glib-1.0.so.0
libk5crypto.so.3
libkeyutils.so.1
libkrb5.so.3
libkrb5support.so.0
liblz4.so.1
liblzma.so.5
libmount.so.1
libm.so.6
libpango-1.0.so.0
libpangocairo-1.0.so.0
libpangoft2-1.0.so.0
libpcre.so.1
libpixman-1.so.0
libpng16.so.16
libpsl.so.5
libpthread.so.0
libresolv.so.2
librt.so.1
libselinux.so.1
libsoup-2.4.so.1
libsqlite3.so.0
libsystemd.so.0
libthai.so.0
libunistring.so.2
libunwind.so.8
libuuid.so.1
libwayland-client.so.0
libwayland-cursor.so.0
libwayland-egl.so.1
libX11.so.6
libXau.so.6
libxcb-render.so.0
libxcb-shm.so.0
libxcb.so.1
libXcomposite.so.1
libXcursor.so.1
libXdamage.so.1
libXext.so.6
libXfixes.so.3
libXinerama.so.1
libXi.so.6
libxkbcommon.so.0
libxml2.so.2
libXrandr.so.2
libXrender.so.1
libz.so.1
linux-vdso.so.1

That's a lot more shared libraries, so we'll always be showing as using a lot of memory in some ways. But which of those do we specifically depend on?

libatk-1.0.so.0
libcairo-gobject.so.2
libcairo.so.2
libc.so.6
libgdk-3.so.0
libgdk_pixbuf-2.0.so.0
libgio-2.0.so.0
libglib-2.0.so.0
libgobject-2.0.so.0
libgspell-1.so.2
libgstreamer-1.0.so.0
libgtk-3.so.0
libjson-glib-1.0.so.0
libm.so.6
libpango-1.0.so.0
libsoup-2.4.so.1
libsqlite3.so.0

From there, we have:

• libsoup pulls in Kerberos support, XML and some unicode/idn handling
• libgtk pulls in about 12 libraries, which pull in at least a dozen more, and a couple more levels of dependency. Including a lot of the freetype and compression libraries
• libgdk pulls in 9 libraries, almost exclusively X/Wayland
• libcairo pulling in xcb renderers and various GL-related bits
• several other libraries with dependencies

But 90% of those are probably used by at least one other app. So while they show up in the RSS and VSZ values and we'd be using that much memory on our own if we were the only app running, the reality isn't anywhere near that simple.

So, something like Firefox might look like it's not using as much RAM, but a good chunk of that is probably because it's not pulling in quite so many libraries that pull in other libraries that pull in…. But those are shared libraries, so it's not just our memory.

(But I think we still have other issues around image handling that could be done better)

[bbhtt]

But which version of "memory usage" is that using? RSS (resident set size - current actual memory usage) or VSZ (virtual size - effectively the potential memory usage, including stuff it's not using)

It actually reports RSS size but it is optimised for python based applications and uses different backend depending on OS. They don't have a lot of documentation on the project page regarding this. I tried to use Valgrind to report memory usage but according to online sources it slows down the process, so Cawbird started lagging and thus any memory usage reported by it will not be accurate.

But 90% of those are probably used by at least one other app. So while they show up in the RSS and VSZ values and we'd be using that much memory on our own if we were the only app running, the reality isn't anywhere near that simple.

Yeah I was reading that https://utcc.utoronto.ca/~cks/space/blog/linux/LinuxMemoryStats, there isn't a way to isolate actual memory used by the process and by its shared libraries? And RSS/VSZ overcounts memory since they can include double entries of code for a single process is using along with other shared things.

[IBBoard]

Latest numbers after the updates to how we load images, based on "run Cawbird with /usr/bin/time -f "RSS: %MkiB" build/cawbird, let it complete and then close it".

• Before update: 360MB
• After update: 220MB
• After update, image loading disabled: 130MB

These numbers are just representative figures. They'll vary depending on your latest timeline and how many images there are.

[IBBoard]

Okay, that should be a good bit of optimisation. We'll still slowly eat memory if people have "scroll on new tweets" disabled and don't look at it for hours on end, but there's not much we can do about that without getting far more complex (and less responsive for many people, and prone to issues) with "load as it comes in to view" behaviour.