Sanity check memory use in 5.0.0

[tannewt]

re: https://forums.adafruit.com/viewtopic.php?f=60&t=162220

A hackspace magazine example no longer works due to 30k more memory use.

Would someone be willing to replicate this? Thanks!

[ladyada]

@jepler wanna take a look?
im suspicious of the bitmap, maybe it wasnt converted to 8-bit indexed?

[jepler]

@ladyada I can take a look on Tuesday if that's soon enough

[ladyada]

yep!

[dhalbert]

I started looking at this. I get the identical memory issue when the program runs automatically as code.py. If I import code.py in the REPL it runs fine.

otter_data.py has a hideously large set of nested lists of small integers and could be made a lot more efficient.

Here is an instrumented view of memory consumption:

import board
import gc
print("before most imports", gc.mem_free())
import displayio
print("a", gc.mem_free())
from time import sleep
import otter_data
print("b", gc.mem_free())
display = board.DISPLAY
group = displayio.Group()
print("c", gc.mem_free())
otter_bitmap = displayio.Bitmap(384,96,18)   # <---- fails here
print("d", gc.mem_free())
background_file = open("/images/background.bmp", "rb")
background_bitmap = displayio.OnDiskBitmap(background_file)
background_tile = displayio.TileGrid(background_bitmap, pixel_shader=displayio.ColorConverter())
#...

Automatic run:

Auto-reload is on. Simply save files over USB to run them or enter REPL to disable.
code.py output:
before most imports 220080
a 220064
b 32000
c 31904
Traceback (most recent call last):
  File "code.py", line 13, in <module>
MemoryError: memory allocation failed, allocating 36864 bytes

Manual import of code.py:

Adafruit CircuitPython 5.0.0-beta.5 on 2020-02-05; Adafruit PyPortal with samd51j20
>>> import code
before most imports 219744
a 219728
b 70336
c 70240
d 33328

[dhalbert]

I bisected and got to this commit as the first one that causes the problem: d32dc81, which is #2330, which fixes a GC bug in displayio, but supposedly only for epaper, and the code is clearly meant only for epaper. I repeated the bisect and got the same thing. Moving back one commit, I don't see the issue. So this would seem to make no sense, because I'm testing with a pyportal, which is not epaper. But maybe there is a mixup somewhere in displayio about epaper vs not?

Tagging @tannewt in case this rings a bell. Taking a break now.

[tannewt]

Nothing jumps out at me. My next step would be to see if I can break in that collect via a debugger. Let me know if you'd like me to take a look.

[dhalbert]

Yes, I'll look at this tomorrow morning: set some breakpoints and see what's going on. It shouldn't be calling the epaper code at all.

[dhalbert]

This was not working on beta.5, but is working on tip of master now. 🙃 More later.

[tannewt]

Are we using the inline limits at all on this board? I wonder if the compiler is improperly dropping the type check sometimes.

[dhalbert]

No, pyportal has no tricky stuff in mpconfigboard.*

[dhalbert]

I am doing some more bisects, but I'm suspicious of something uninitialized somewhere.

[dhalbert]

I did a bisect from 5.0.0-beta.5 to the tip of master. git bisect assumes you are going from "good" to "bad", not the other way, so I had to mark good results as bad and vice versa. (I didn't know this at first and got into some confusing situations.) The problem got "fixed" at 36e6cc8 (#2614), which makes sense because it allocated memory more efficiently, but it didn't necessarily fix any underlying bug. Will do a little more looking on beta.5.