Possible memory leak when reading data

[elisalle]

When repeatedly reading the same data from a netCDF via mdal-python, the memory usage on the computer rises until it is full, at which point the computer crashes. The netCDF I was reading is here: https://nelenschuurmans-my.sharepoint.com/:u:/g/personal/eli_salle_nelen-schuurmans_nl/EQF4yujI03hNi_9gFPd8L7gBDWmAseLXkCBrKSWy5gK9dg?e=ouudP9
My code is as follows:

from mdal import Datasource
from pathlib import Path
import gc

mypath = "results_3di.nc"
ds = Datasource(mypath)

with ds.load() as mesh:
    print("mesh loaded")
    group = mesh.group(3)
    for i in range(30):
        for i in range(0, group.dataset_count):
            data = group.data(i)
            # make very sure the variable gets taken out of memory:
            del(data)
            gc.collect()

I also tried exiting the context manager on each loop to see if that would help, but it made no difference:

from mdal import Datasource
from pathlib import Path
import gc

mypath = "results_3di.nc"
ds = Datasource(mypath)

for i in range(30):
    with ds.load() as mesh:
        print("mesh loaded")
        group = mesh.group(3)
        for i in range(0, group.dataset_count):
            data = group.data(i)
            # make very sure the variable gets taken out of memory:
            del(data)
            gc.collect()

[runette]

Yes - I have had the same problem. There is a mistake in the lifecycle of DataGroup object!

I have been working on some updates - I will try to complete them in the next week or so - I have a little - and I will test them on your file to see if it fixes the problem.

[runette]

I have created a fix - it was not the problem that I thought it was. It took an interesting turn through the details of Python object reference counts but it is (hopefully fixed now)

Fixed in Version 1.0.2 - should be rolling out to distributions over the next day or so

[elisalle]

Thanks for the quick fix.