currentNcFiles memory usage

[simonbyrne]

I just tracked down a large memory leak to the existence of currentNcFiles: while the fix is easy (calling ncclose after reading from each file), the overall approach of keeping a global reference to each opened file is a bit strange. Any reason why this is done?

[visr]

Yes getting rid of those global references is an important part of #61. It is a pretty major overhaul, and has gone stale and needs rebasing. If anybody feels like picking that up, that would be great.

[meggart]

I agree this should have high priority to be fixed, and maybe just picking the parts that remove the currentNcFiles stuff from #61 as a smaller PR might be not too much work. I am not so full this week and will give it a try.

[natgeo-wong]

Hello! I've run into a similar error (although I didn't know it was this at first I went one big round into logging functions), because I handle the downloading and extraction of data from many NetCDF files in my Julia code.

I'm currently converting my MATLAB and Python scripts to Julia, and it's a bit jarring that both MATLAB and Python seem to have inbuilt "ncclose()" functionalities within ncread, if you will, whereas this is not the case for Julia.

Is this issue still being looked into at the moment?

[visr]

Hmm this is already fixed in #95, but I see we haven't tagged a release yet. Should do so shortly.