Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make sure __getitem__ always returns a numpy array #2399

Merged
merged 2 commits into from
May 12, 2023

Conversation

jfoster17
Copy link
Member

Pull Request Template

Description

In some circumstances, dask.compute() can still return a dask array (rather than the desired numpy array). I was able to trigger this with an external data loader library that presumably falls foul of this bug. Explicit to np.array fixes this for my use case and seems otherwise totally innocuous.

I am not providing a new unit test because I'm not sure how to reliably reproduce the extra-lazy dask array outside of the specific data loading library I was using.

Fixes #2397

@jfoster17 jfoster17 requested a review from astrofrog May 8, 2023 21:16
Copy link
Member

@astrofrog astrofrog left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for tracking this down! Just a small comment below.

@@ -536,7 +536,7 @@ def ndim(self):
return len(self._data.shape)

def __getitem__(self, key):
return self._data[key].compute()
return np.array(self._data[key].compute())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be better to use np.asarray in case the result of the computation is already a Numpy array, otherwise it will be unnecessarily copied in this case.

@astrofrog astrofrog merged commit d197670 into glue-viz:main May 12, 2023
@jfoster17 jfoster17 deleted the fix-lazy-dask-components branch June 9, 2023 15:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

compute_fixed_resolution_buffer fails with some components powered by dask arrays
2 participants