GPU Memory leak?

[smumriak]

After running for ~30 minutes on RPi 3B with 96MB GPU memory allocated i've started getting message "Failed to allocate GPU memory! Try increasing gpu_mem allocation in /boot/config.txt. See https://www.raspberrypi.org/documentation/configuration/config-txt/memory.md".
I don't actually think it's a memory leak, but maybe some issue in memory mapping calculation?

Video setup in config.txt:
gpu_mem=96 hdmi_force_hotplug=1 hdmi_group=2 hdmi_mode=87 hdmi_cvt 480 320 60 6 0 0 0

Built parameters:
-DSPI_BUS_CLOCK_DIVISOR=12 -DWAVESHARE35B_ILI9486=ON -DARMV6Z=ON

No overclocking/downclocking.

[juj]

That memory allocation failure message comes from line 145 at

fbcp-ili9341/dma.cpp

Lines 136 to 153 in 2393c11

	// Allocates the given number of bytes in GPU side memory, and returns the virtual address and physical bus address of the allocated memory block.
	// The virtual address holds an uncached view to the allocated memory, so writes and reads to that memory address bypass the L1 and L2 caches. Use
	// this kind of memory to pass data blocks over to the DMA controller to process.
	GpuMemory AllocateUncachedGpuMemory(uint32_t numBytes, const char *reason)
	{
	GpuMemory mem;
	mem.sizeBytes = ALIGN_UP(numBytes, PAGE_SIZE);
	uint32_t allocationFlags = MEM_ALLOC_FLAG_DIRECT | MEM_ALLOC_FLAG_COHERENT;
	mem.allocationHandle = Mailbox(MEM_ALLOC_MESSAGE, /*size=*/mem.sizeBytes, /*alignment=*/PAGE_SIZE, /*flags=*/allocationFlags);
	if (!mem.allocationHandle) FATAL_ERROR("Failed to allocate GPU memory! Try increasing gpu_mem allocation in /boot/config.txt. See https://www.raspberrypi.org/documentation/configuration/config-txt/memory.md");
	mem.busAddress = Mailbox(MEM_LOCK_MESSAGE, mem.allocationHandle);
	if (!mem.busAddress) FATAL_ERROR("Failed to lock GPU memory!");
	mem.virtualAddr = mmap(0, mem.sizeBytes, PROT_READ | PROT_WRITE, MAP_SHARED, mem_fd, BUS_TO_PHYS(mem.busAddress));
	if (mem.virtualAddr == MAP_FAILED) FATAL_ERROR("Failed to mmap GPU memory!");
	totalGpuMemoryUsed += mem.sizeBytes;
	// printf("Allocated %u bytes of GPU memory for %s (bus address=%p). Total GPU memory used: %llu bytes\n", mem.sizeBytes, reason, (void*)mem.busAddress, totalGpuMemoryUsed);
	return mem;
	}

The only place where that AllocateUncachedGpuMemory() is called is from InitDMA() in

fbcp-ili9341/dma.cpp

Lines 216 to 217 in 2393c11

	int InitDMA()
	{

which is called exactly once at application startup, but never at runtime. Exactly three calls to AllocateUncachedGpuMemory() are made.

It is odd if you are seeing this error appear during runtime. Are you shutting down and restarting fbcp-ili9341 multiple times while the Pi is running?

The deallocation function for those uncached GPU memory blocks is at

fbcp-ili9341/dma.cpp

Lines 729 to 735 in 2393c11

	void DeinitDMA(void)
	{
	WaitForDMAFinished();
	ResetDMAChannels();
	FreeUncachedGpuMemory(dmaSourceBuffer);
	FreeUncachedGpuMemory(dmaCb);
	FreeUncachedGpuMemory(dmaConstantData);

where the corresponding three memory blocks are freed.

DeinitDMA() is called from DeinitSPI(), which is called at the end of the program at

fbcp-ili9341/fbcp-ili9341.cpp

Lines 562 to 568 in 2393c11

	DeinitGPU();
	DeinitSPI();
	CloseMailbox();
	CloseKeyboard();
	printf("Quit.\n");
	}

.

If fbcp-ili9341 shuts down cleanly, it should free the GPU memory. There are a couple of places in the program where exit(1) is called directly without shutting down cleanly. Perhaps one of those is being called during execution that is causing GPU memory to be leaked behind? Try to debug if you are getting an unclean shutdown of the application.

[juj]

Adjusted the code at 83f1326 to more gracefully shut down if DispmanX API fails. If that is the cause, then it is possible that the new code will properly free up GPU resources.

[smumriak]

I should add more input:
I am running fbcp as a systemd-service. This error comes after relaunch of fbcp after it has crashed. After some tests i found that memory is indeed not unmapped at any point. After some time something inside gpu.cpp exitst via FATAL_ERROR. I'll pull your fresh changes and will make more tests.

[juj]

Reading the FATAL_ERROR does suggest like the above commit could fix the error. This line 83f1326#diff-097f3581ebc42989986bca651ddc68c4R126 is a known place where Raspberry Pi kernel code trips up due to this bug in system DispmanX: raspberrypi/userland#461 that leaves no options to fbcp-ili9341 than to abort. The above commit changed it so that when that happens, it should cleanly free up system resources, so when running as a service, it should no longer leak.

I'll close this bug based on the assumption that the above machinery is working, though feel free to reopen if you still are encountering the issue.