Releasing memory from deleted Objc classes

[samschott]

Describe the bug
I am not sure if this actually is a bug or intended behaviour. When deleting memory heavy objects on the Python side, the actual memory is not released / returned to the OS. I know that Python sometimes does not give back freed memory, especially in case of small blocks which may be reused later. But this seems different.

Is rubicon-objc hanging on to a reference of those objects somewhere? Or does ObjC just not release the memory when all Python references have disappeared?

To Reproduce
This can be reproduced by loading a 14 MB image into memory and deleting the Python reference to the object:

from rubicon.objc import ObjCClass
from rubicon.objc.runtime import load_library

appkit = load_library("AppKit")

NSImage = ObjCClass("NSImage")

# load a 14 MB image into memory
image = NSImage.alloc().initWithContentsOfFile("/System/Library/Desktop Pictures/Catalina Coast.heic")

del image  # memory is not released

Expected behavior
I would have expected at least part the 14 MB to be returned to the OS.

Environment:

• Rubicon version: 0.4.1.dev1
• Operating System: macOS 11
• Python version: 3.9

[dgelessus]

This is technically not a bug, but definitely not ideal behavior. At the moment, rubicon-objc simply doesn't do any automatic Objective-C memory management. This means that all Objective-C calls follow the old rules for manual memory management: every object returned from an alloc/new/copy method will have an extra reference, and it's the caller's responsibility to manually release that reference once they're done with the object.

So in this case, because image is returned from a .alloc().init...(...) call, it has an extra reference, which you would have to manually release using image.release() before the del image. Or depending on how the object is used in your actual code, it may be easier to use .autorelease() right when the object is created, to keep the object alive until the current autorelease pool is exited. (Autorelease pools currently also can't be managed easily from Rubicon, but this normally isn't a problem, because AppKit automatically provides an autorelease pool for code called from the event loop).

It would definitely be better though if rubicon-objc would take care of all retaining and releasing automatically. (See also #48 for some previous discussion on this.) Implementing this shouldn't even be very difficult (it would probably only need some calls to retain and release in ObjCInstance's __new__ and __del__), it's just that nobody has actually done it yet 🙂

Off-topic: I see you're on macOS 11 - I take it that rubicon-objc is working normally so far on the new OS version? If you run into any compatibility issues because of macOS 11, please let us know 🙂 (especially if you happen to have one of the new ARM-based Macs!)

[dgelessus]

btw, if anyone is wondering why I remove and re-add labels on every new issue: GitHub's handling of labels with organization-wide issue templates seems to be kinda broken. The rubicon-objc repo doesn't have its own set of issue templates, so it automatically uses the templates from the beeware/.github repo instead. These templates also include default issue labels - for example, issues created with the bug report template automatically get the "bug" label. But for some reason, it does this by assigning the new issue the actual "bug" label from beeware/.github, which is not the same thing as the "bug" label from beeware/rubicon-objc! This leads to fun issues, like an issue search for "is:issue is:open label:bug" possibly not showing all issues labeled "bug" - because some of them have a "bug" label from a different repo.

You can see this if you look at my label change above. The label in "added bug" (on the left, with black text) is the correct label from the rubicon-objc repo, and the label in "removed bug" (on the right, with white text) is the one from the beeware/.github repo.

[samschott]

Thank you @dgelessus for the detailed explanation. It appears as if toga does not attempt to release garbage collected native widgets and therefore does not free memory for widgets that are only temporary. Writing an image viewer in toga for instance would be a really bad idea at the moment.

[samschott]

Regarding macOS 11, I am seeing no issues at all so far. One of the larger changes is how libraries are loaded from the shared cache. But this has been addressed in beeware/toga#1091 and rubicon itself always worked.

Unfortunately I am not on an ARM mac, so I cannot help you there.