Skip to content

Commit

Permalink
concat would load dask variables if first argument was numpy-based (#…
Browse files Browse the repository at this point in the history
  • Loading branch information
crusaderky authored and shoyer committed Sep 25, 2017
1 parent dc7d733 commit 3a91442
Show file tree
Hide file tree
Showing 3 changed files with 11 additions and 3 deletions.
4 changes: 4 additions & 0 deletions doc/whats-new.rst
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,10 @@ Bug fixes
when objects other than ``Dataset`` are provided (:issue:`1555`).
By `Joe Hamman <https://github.com/jhamman>`_.

- :py:func:`xarray.concat` would eagerly load dask variables into memory if
the first argument was a numpy variable (:issue:`1588`).
By `Guido Imperiale <https://github.com/crusaderky>`_.

.. _whats-new.0.9.6:

v0.9.6 (8 June 2017)
Expand Down
8 changes: 5 additions & 3 deletions xarray/core/duck_array_ops.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,9 +40,11 @@ def _dask_or_eager_func(name, eager_module=np, list_of_args=False,
"""Create a function that dispatches to dask for dask array inputs."""
if has_dask:
def f(*args, **kwargs):
dispatch_args = args[0] if list_of_args else args
if any(isinstance(a, da.Array)
for a in dispatch_args[:n_array_args]):
if list_of_args:
dispatch_args = args[0]
else:
dispatch_args = args[:n_array_args]
if any(isinstance(a, da.Array) for a in dispatch_args):
module = da
else:
module = eager_module
Expand Down
2 changes: 2 additions & 0 deletions xarray/tests/test_dask.py
Original file line number Diff line number Diff line change
Expand Up @@ -178,6 +178,8 @@ def test_concat(self):
v = self.lazy_var
self.assertLazyAndIdentical(u, Variable.concat([v[:2], v[2:]], 'x'))
self.assertLazyAndIdentical(u[:2], Variable.concat([v[0], v[1]], 'x'))
self.assertLazyAndIdentical(u[:2], Variable.concat([u[0], v[1]], 'x'))
self.assertLazyAndIdentical(u[:2], Variable.concat([v[0], u[1]], 'x'))
self.assertLazyAndIdentical(
u[:3], Variable.concat([v[[0, 2]], v[[1]]], 'x', positions=[[0, 2], [1]]))

Expand Down

0 comments on commit 3a91442

Please sign in to comment.