qgis/gdal garbage collection causes segfault

[velle]

What is the bug or the crash?

When opening a dataset with gdal, and not explicitly dereferencing it, a segmentation fault will happen when QGIS terminates.

I had an email correspondence with @rouault where he explained the following:

that's a bad interaction between QGIS, GDAL and Python "garbage collection".

The issue is that in this situation QGIS runs its termination before the
"dataset" object is released, and QGIS termination calls the GDAL
termination logic, the GDAL termination logic will force closed still
opened datasets, so when Python garbage collects dataset and run
GDALClose() on it, this results in a double free attempt and a crash.

A workaround is to explicitly drop the reference to the GDAL dataset
with "del dataset" or "dataset = None" before the Python function exits.

I asked if this was to be considered a bug and he responded:

Maybe when using startApp() from Python, QGIS should set some flag to prevent the 2 below lines to be run:

QGIS/src/core/qgsapplication.cpp

Lines 1638 to 1639 in b0b3736

	OGRCleanupAll();
	GDALDestroyDriverManager();

Running GDAL termination is not strictly needed. It just nicer when running under memory analyzers that could otherwise point to unreleased memory, but in a Python testing use case, we probably don't care about that.

Maybe there's a better solution, but can't think of it right now.

Steps to reproduce the issue

Create a Python script with the following code, and run as a stand alone.

#!/usr/bin/env python3
# segfault.py
from osgeo import gdal
from osgeo.gdalconst import GA_ReadOnly
from qgis.testing import start_app

start_app()
dataset = gdal.Open('/home/velle/a/QGIS/tests/testdata/raster/rgb_with_mask.tif', GA_ReadOnly)
print('the end')

This gives:

the end
Segmentation fault

If both of these lines are left, it gives segfault:

• start_app()
• gdal.Open(...)

If any of them are commented out, the segfault disappears.

I tried with a few different tif, and the result is the same.

If changing the order of the two lines, some times it results in segfault, and sometimes it results in the following fatal error:

the end
double free or corruption (!prev)
Aborted

The print line is always executed before the fatal error occurs.

Versions

qgis master (Sep 13 2024)
gdal 3.4.1

Supported QGIS version

• I'm running a supported QGIS version according to the roadmap.

New profile

• I tried with a new QGIS profile

Additional context

No response