Chunk issue with sdba.MBCn.train #1903
Comments
|
It seems to be related to the group attribute. I do not face this error when I use |
|
Hi!
Are you sure? That doesn't seem to be the case in the code snippet you shared, for your historical dataset you have
Try running Cheers, |
|
Hello Éric, I get: Below is how my inputs look like. dhist dsim ADJ.ds Is it OK to use proleptic gregorian calendar and 2 dimensions for space? |
|
No worries! Space dimensions are not an issue. I will look into it, but if you're using Can you try: and see if this removes the error messages? In the meantime I will investigate. |
|
Can you share more information on the error? If you could share 3 .nc files for |
|
I still have the same error message after converting calendars. Please find my netCDF files as well as my python script at (.py and .nc files are not supported): |
|
The python script in your folder is for the old way of working things with |
|
I have updated the file. |
|
Thanks! The problem is that you have different time coordinates: for instance if you try dhist["time"] = dref.timethe error disappears. I think we could control this earlier in the process since our methods expect equal time coordinates and give a clearer error message. Thanks for this feedback. Let me know if there are other problems after that. |
|
For xclim devs: We could have a similar function to |
Generic Issue
Xclim version: 0.52.0
Python version: 3.10.14
Operating System: Linux
Description
I apply the MBCn algorithm for bias-correcting Euro CORDEX data. I successfully reproduced the example presented in the notebook. Now I follow the same procedure using my data.
What I Did
I loaded my files using open_mfdataset and merged them (one file per variable) to create dref and dhist. The reference and historical datasets are made of a single chunk of size 7305 for the time coordinate.
This is my reference file:
This is my historical file:
What I Received
I got the following error message when I executed:
File /.../conda/envs/xclim_metpy/lib/python3.10/site-packages/xarray/core/computation.py:767, in apply_variable_ufunc(func, signature, exclude_dims, dask, output_dtypes, vectorize, keep_attrs, dask_gufunc_kwargs, *args)
765 for axis, dim in enumerate(core_dims, start=-len(core_dims)):
766 if len(data.chunks[axis]) != 1:
--> 767 raise ValueError(
768 f"dimension {dim} on {n}th function argument to "
769 "apply_ufunc with dask='parallelized' consists of "
770 "multiple chunks, but is also a core dimension. To "
771 "fix, either rechunk into a single array chunk along "
772 f"this dimension, i.e.,
.chunk(dict({dim}=-1)), or "773 "pass
allow_rechunk=Trueindask_gufunc_kwargs"774 "but beware that this may significantly increase memory usage."
775 )
776 dask_gufunc_kwargs["allow_rechunk"] = True
778 output_sizes = dask_gufunc_kwargs.pop("output_sizes", {})
ValueError: dimension time on 1th function argument to apply_ufunc with dask='parallelized' consists of multiple chunks, but is also a core dimension. To fix, either rechunk into a single array chunk along this dimension, i.e.,
.chunk(dict(time=-1)), or passallow_rechunk=Trueindask_gufunc_kwargsbut beware that this may significantly increase memory usage.What is going wrong here?
Code of Conduct
The text was updated successfully, but these errors were encountered: