Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

recognize _Encoding attribute for char and string arrays #665

Merged
merged 29 commits into from
May 19, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
bd91f93
get rid of 'default_encoding' and 'unicode_error' module variables.
jswhit May 16, 2017
7a8779c
add encoding kwarg to getncattr
jswhit May 16, 2017
659addf
make sure all vlen string vars use _Encoding
jswhit May 16, 2017
913dd95
use chartostring if _Encoding specified for a NC_CHAR variable
jswhit May 16, 2017
ae1beed
use stringtochar to write array of strings with _Encoding is set for
jswhit May 16, 2017
46d9e41
fix bug in previous commit
jswhit May 16, 2017
11ee7f9
use errors='surrogateescape' in chartostring and stringtochar,
jswhit May 16, 2017
52d8f7c
regenerate C source
jswhit May 16, 2017
6f4b66c
don't use 'surrogateescape' since it's not available in python 2.7
jswhit May 16, 2017
2ffb32f
update
jswhit May 16, 2017
371b34b
fix failing tests
jswhit May 16, 2017
047ef12
get nc_open_mem from netcdf_mem.h
jswhit May 17, 2017
0e75433
regenerate C source
jswhit May 17, 2017
74b0353
add test for _Encoding with char arrays
jswhit May 17, 2017
fb49b45
use ascii not utf-8
jswhit May 17, 2017
7f6204a
add some different slices to test
jswhit May 17, 2017
083342e
add new test for unicode attributes
jswhit May 17, 2017
779ac4a
set NO_NET=1
jswhit May 17, 2017
7748778
set NO_NET=1 in run_test.py
jswhit May 17, 2017
ac4157a
update
jswhit May 17, 2017
da9fc72
make sure python strings are cast to a numpy string array when writing
jswhit May 17, 2017
048d590
update
jswhit May 17, 2017
42e2643
test writing a python string, whether _Encoding is ignored when an
jswhit May 17, 2017
6d76a26
perform conversion for bytes too
jswhit May 17, 2017
8078e60
fix failing test in python 3
jswhit May 17, 2017
a009936
don't convert to stringarr if last dimension of slice doesn't match
jswhit May 18, 2017
eb9ff0e
fix corner case when slice is not along last dimension (don't return a
jswhit May 18, 2017
0d9ade9
add set_auto_chartostring Dataset and Variable method.
jswhit May 18, 2017
6d50a22
update docstrings
jswhit May 18, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions Changelog
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,20 @@
(pull request #652, issues #406 and #295).
* fix for negative times in num2date (issue #659).
* fix for failing tests in numpy 1.13 due to changes in numpy.ma (issue #662).
* Checking for _Encoding attribute for NC_STRING variables, otherwise use
'utf-8'. 'utf-8' is used everywhere else, 'default_encoding' global module
variable is no longer used. getncattr method now takes optional kwarg
'encoding' (default 'utf-8') so encoding of attributes can be specified
if desired. If _Encoding is specified for an NC_CHAR ('S1') variable,
the chartostring utility function is used to convert the array of
characters to an array of strings with one less dimension (the last
dimension is interpreted as the length of each string) when reading the
data. When writing the data, stringtochar is used to convert a numpy
array of fixed length strings to an array of characters with one more
dimension. chartostring and stringtochar now also have an 'encoding' kwarg.
Automatic conversion to/from character to string arrays can be turned off
via a new set_auto_chartostring Dataset and Variable method (default
is True).

version 1.2.7 (tag v1.2.7rel)
==============================
Expand Down
10 changes: 2 additions & 8 deletions conda.recipe/run_test.py
Original file line number Diff line number Diff line change
@@ -1,14 +1,8 @@
import os
import netCDF4

# Check OPeNDAP functionality.
url = 'http://geoport-dev.whoi.edu/thredds/dodsC/estofs/atlantic'
nc = netCDF4.Dataset(url)

# Check if it was compiled with cython.
assert nc.filepath() == url

# Run the unittests.
# Run the unittests, skipping the opendap test.
test_dir = os.path.join('test')
os.chdir(test_dir)
os.environ['NO_NET']='1'
os.system('python run_all.py')
252 changes: 215 additions & 37 deletions docs/netCDF4/index.html

Large diffs are not rendered by default.

6 changes: 5 additions & 1 deletion include/netCDF4.pxi
Original file line number Diff line number Diff line change
Expand Up @@ -250,7 +250,6 @@ cdef extern from "netcdf.h":
int nc_create(char *path, int cmode, int *ncidp)
int nc__create(char *path, int cmode, size_t initialsz, size_t *chunksizehintp, int *ncidp)
int nc_open(char *path, int mode, int *ncidp)
int nc_open_mem(const char *path, int mode, size_t size, void* memory, int *ncidp)
int nc__open(char *path, int mode, size_t *chunksizehintp, int *ncidp)
int nc_inq_path(int ncid, size_t *pathlen, char *path) nogil
int nc_inq_format_extended(int ncid, int *formatp, int* modep) nogil
Expand Down Expand Up @@ -718,6 +717,11 @@ cdef extern from "netcdf.h":

int nc_inq_enum_ident(int ncid, nc_type xtype, long long value, char *identifier) nogil


IF HAS_NC_OPEN_MEM:
cdef extern from "netcdf_mem.h":
int nc_open_mem(const char *path, int mode, size_t size, void* memory, int *ncidp)

# taken from numpy.pxi in numpy 1.0rc2.
cdef extern from "numpy/arrayobject.h":
ctypedef int npy_intp
Expand Down
28,197 changes: 14,329 additions & 13,868 deletions netCDF4/_netCDF4.c

Large diffs are not rendered by default.

Loading