You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Found while working on #2484: PEcAn.BIOCRO::met2model.BIOCRO loops over as many years of CDF inputs as you ask it to process, but only calls nc_close once at function exit. When processing more than one year, this results in the filehandles for all but the last year being left open.
Patch incoming, just documenting it here for posterity.
To Reproduce
Edit paths freely -- I'm showing with GFDL because it's what I used, but results should be the same with any multi-year data source you have handy.
In an R session that will stay alive after function exit (i.e. not just Rscript -e):
In another terminal, with the above R session still active (so that we see the leaks before they're cleaned up by the OS when R exits):
lsof -c R -a +D ~/pecan/dbfiles/
Expected behavior
$ lsof -c R -a +D ~/pecan/dbfiles/
$
Observed behavior
$ lsof -c R -a +D ~/pecan/dbfiles/
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
R 97927 chrisb 24r REG 1,4 118164 4363299343 /Users/chrisb/pecan/dbfiles/GFDL_site_0-753/GFDL_CM3_rcp45_r1i1p1_site_0-753/GFDL.CM3.rcp45.r1i1p1.2009.nc
R 97927 chrisb 26r REG 1,4 118164 4363297178 /Users/chrisb/pecan/dbfiles/GFDL_site_0-753/GFDL_CM3_rcp45_r1i1p1_site_0-753/GFDL.CM3.rcp45.r1i1p1.2010.nc
$
Proposed fix
Explicitly call ncdf4::nc_close at the end of each year's processing. We should keep closing the file in on.exit to make sure we aren't leaking filehandles when the function terminates early, but will need to do it conditionally to avoid ncdf4 complaining when we try to close files that were closed already.
The text was updated successfully, but these errors were encountered:
Bug Description
Found while working on #2484: PEcAn.BIOCRO::met2model.BIOCRO loops over as many years of CDF inputs as you ask it to process, but only calls
nc_close
once at function exit. When processing more than one year, this results in the filehandles for all but the last year being left open.Patch incoming, just documenting it here for posterity.
To Reproduce
Edit paths freely -- I'm showing with GFDL because it's what I used, but results should be the same with any multi-year data source you have handy.
In an R session that will stay alive after function exit (i.e. not just
Rscript -e
):In another terminal, with the above R session still active (so that we see the leaks before they're cleaned up by the OS when R exits):
Expected behavior
Observed behavior
Proposed fix
Explicitly call
ncdf4::nc_close
at the end of each year's processing. We should keep closing the file inon.exit
to make sure we aren't leaking filehandles when the function terminates early, but will need to do it conditionally to avoid ncdf4 complaining when we try to close files that were closed already.The text was updated successfully, but these errors were encountered: