You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When reading a dataset containing complex variables using Dataset.open_zarr method the following warning appears:
_/home/.../python3.8/site-packages/xarray/coding/variables.py:218: ComplexWarning: Casting complex values to real discards the imaginary part
And the imaginary part is effectively discarded which is not what I expected.
After a slightly more in-depth analysis I came across the function (xarray/coding/variables.py:226)
def_choose_float_dtype(dtype, has_offset):
"""Return a float dtype that can losslessly represent `dtype` values."""# Keep float32 as-is. Upcast half-precision to single-precision,# because float16 is "intended for storage but not computation"ifdtype.itemsize<=4andnp.issubdtype(dtype, np.floating):
returnnp.float32# float32 can exactly represent all integers up to 24 bitsifdtype.itemsize<=2andnp.issubdtype(dtype, np.integer):
# A scale factor is entirely safe (vanishing into the mantissa),# but a large integer offset could lead to loss of precision.# Sensitivity analysis can be tricky, so we just use a float64# if there's any offset at all - better unoptimised than wrong!ifnothas_offset:
returnnp.float32# For all other types and circumstances, we just use float64.# (safe because eg. complex numbers are not supported in NetCDF)returnnp.float64
For me, this behavior is strange, I find more natural to use the stored type rather than to make a systematic transformation into a float.
To test, I have modified the decode method (xarray/coding/variables.py:265)
and it is working as I expected.
If there is a good reason to keep things as they are, can you explain me how to deal with complex data without creating a new variable?
Thank you for your great job, xarray is awesome.
The text was updated successfully, but these errors were encountered:
I finally found the source of my problem.
The data I read from the zarr store was initially stored in tiff files that I opened using the open_rasterio method. It seems that open_rasterio systematically adds the attributes scale_factor=1 and add_offset=0. By removing them everything work as I expected, so the problem would come from open_rasterio.
We've deleted the internal rasterio backend in favor of rioxarray. If this issue is still relevant, please migrate the discussion to the rioxarray repo
xarray version 0.16.2/ Python 3.8.5
When reading a dataset containing complex variables using Dataset.open_zarr method the following warning appears:
_/home/.../python3.8/site-packages/xarray/coding/variables.py:218: ComplexWarning: Casting complex values to real discards the imaginary part
And the imaginary part is effectively discarded which is not what I expected.
After a slightly more in-depth analysis I came across the function (xarray/coding/variables.py:226)
For me, this behavior is strange, I find more natural to use the stored type rather than to make a systematic transformation into a float.
To test, I have modified the decode method (xarray/coding/variables.py:265)
and it is working as I expected.
If there is a good reason to keep things as they are, can you explain me how to deal with complex data without creating a new variable?
Thank you for your great job, xarray is awesome.
The text was updated successfully, but these errors were encountered: