Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EXP: For big cube, be a minimalist #8

Closed
wants to merge 2 commits into from
Closed

EXP: For big cube, be a minimalist #8

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a266f03
get_data: REMOVE support for function/spatial_subset
kecnry Apr 30, 2024
abe7f40
EXP: For big cube, be a minimalist
pllim May 1, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 11 additions & 3 deletions jdaviz/configs/cubeviz/plugins/parsers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@
__all__ = ['parse_data']

EXT_TYPES = dict(flux=['flux', 'sci', 'data'],
uncert=['ivar', 'err', 'var', 'uncert'],
mask=['mask', 'dq', 'quality'])
uncert=['ivar', 'err', 'error', 'var', 'uncert'],
mask=['mask', 'dq', 'quality', 'data_quality'])


@data_parser_registry("cubeviz-data-parser")
Expand Down Expand Up @@ -214,6 +214,8 @@ def _parse_hdulist(app, hdulist, file_name=None,
if hdu.data is None or not hdu.is_image or hdu.data.ndim != 3:
continue

is_big_cube = hdu.data.size > 8_000_000

data_type = _get_data_type_by_hdu(hdu)
if not data_type:
continue
Expand All @@ -228,6 +230,8 @@ def _parse_hdulist(app, hdulist, file_name=None,
if data_type == 'flux':
wcs = WCS(hdu.header, hdulist)
wcs_sci = wcs
elif is_big_cube:
continue
else:
wcs = wcs_sci

Expand All @@ -249,7 +253,11 @@ def _parse_hdulist(app, hdulist, file_name=None,
if hdu.name != 'PRIMARY' and 'PRIMARY' in hdulist:
metadata[PRIHDR_KEY] = standardize_metadata(hdulist['PRIMARY'].header)

sc = _return_spectrum_with_correct_units(flux, wcs, metadata, data_type, hdulist=hdulist)
if not is_big_cube:
sc = _return_spectrum_with_correct_units(
flux, wcs, metadata, data_type, hdulist=hdulist)
else:
sc = Spectrum1D(flux=flux, wcs=wcs, meta=metadata)
Comment on lines +256 to +260
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will this also skip any necessary units stuff? Can we just set/pass hdulist=None and still use _return_spectrum_with_correct_units or is that still too expensive?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_return_spectrum_with_correct_units creates a new Spectrum1D object. I have not looked into whether it does a copy or what. I was just trying some possible low hanging fruits to see if anything helps. Given the ticket is only 2 points, I didn't dig too deep here. There is a chance we do not have to disable this (and I don't think we want to disable if we can avoid it).


# store original WCS in metadata. this is a hacky workaround for converting subsets
# to sky regions, where the parent data of the subset might have dropped spatial WCS info
Expand Down