Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

free string data internally allocated in netcdf-c using nc_free_string (issue #495) #496

Merged
merged 5 commits into from
Dec 9, 2015
Merged

free string data internally allocated in netcdf-c using nc_free_string (issue #495) #496

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
2f53eca
be sure to call nc_free_string to free memory allocated
jswhit Dec 8, 2015
1edf732
add comment
jswhit Dec 8, 2015
39c32da
update
jswhit Dec 8, 2015
1003de5
now use nc_free_vlens using memcpy trick suggested by @ckhroulev
jswhit Dec 8, 2015
1bb5857
update comment
jswhit Dec 8, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions Changelog
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
version 1.2.2 (not yet released)
================================
* Potential memory leaks fixed by freeing string pointers internally allocated
in netcdf-c using nc_free_string. Also use nc_free_vlens to free space allocated
for vlens inside netcdf-c (issue #495).
* invoke str on filename argument to Dataset constructor, so pathlib
instances can be used (issue #489).
* don't use hardwired NC_MAX_DIMS or NC_MAX_VARS internally to allocate space
Expand Down
1 change: 1 addition & 0 deletions include/netCDF4.pxi
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -704,6 +704,7 @@ cdef extern from "netcdf.h":
void nc_set_log_level(int new_level)
int nc_show_metadata(int ncid)
int nc_free_vlen(nc_vlen_t *vl)
int nc_free_vlens(size_t len, nc_vlen_t *vl)
int nc_free_string(size_t len, char **data)
int nc_set_chunk_cache(size_t size, size_t nelems, float preemption)
int nc_get_chunk_cache(size_t *sizep, size_t *nelemsp, float *preemptionp)
Expand Down
18,656 changes: 9,335 additions & 9,321 deletions netCDF4/_netCDF4.c
View file
Open in desktop

Large diffs are not rendered by default.

12 changes: 10 additions & 2 deletions netCDF4/_netCDF4.pyx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -937,6 +937,7 @@ import warnings
from glob import glob
from numpy import ma
from numpy import __version__ as _npversion
from libc.string cimport memcpy
if _npversion.split('.')[0] < '1':
raise ImportError('requires numpy version 1.0rc1 or later')
import_array()
Expand Down Expand Up @@ -1109,6 +1110,7 @@ cdef _get_att(grp, int varid, name):
with nogil:
ierr = nc_get_att_string(_grpid, varid, attname, &stratt)
pstring = stratt.decode(default_encoding,unicode_error).replace('\x00','')
ierr = nc_free_string(1, &stratt) # free memory in netcdf C lib
return pstring
else:
raise KeyError('vlen string array attributes not supported')
Expand Down Expand Up @@ -4338,6 +4340,9 @@ The default value of `mask` is `True`
data[i] = strdata[i].decode(default_encoding)
# reshape the output array
data = numpy.reshape(data, shapeout)
# free string data internally allocated in netcdf C lib
ierr = nc_free_string(totelem, strdata)
# free the pointer array
free(strdata)
else:
# regular vlen
Expand All @@ -4361,11 +4366,14 @@ The default value of `mask` is `True`
for i from 0<=i<totelem:
arrlen = vldata[i].len
dataarr = numpy.empty(arrlen, self.dtype)
dataarr.data = <char *>vldata[i].p
#dataarr.data = <char *>vldata[i].p
memcpy(<void*>dataarr.data, vldata[i].p, dataarr.nbytes)
data[i] = dataarr
# reshape the output array
data = numpy.reshape(data, shapeout)
# free the pointer array.
# free vlen data internally allocated in netcdf C lib
ierr = nc_free_vlens(totelem, vldata)
# free the pointer array
free(vldata)
free(startp)
free(countp)
Expand Down