Use zarrita rather than zarr.js

[keller-mark]

Fixes #

Background

• SpatialData examples contain arrays of <i8 integers. ZarrJS does not support these Support BigInt64Array gzuidhof/zarr.js#143
• Zarr v3 is coming and zarrita supports this
• ZarrJS has a large bundle size, in part due to support for writing, which we do not need. Zarrita is tree-shakeable.
• Zarrita supports decoding string arrays stored as vlen-utf8, so we can get rid of that code here
• ZarrJS does not support unicode string arrays with dtype like <U7
• ZarrJS does not support zero-dimensional array access (e.g., scalar value stored as Zarr with shape [])

Change List

• Wrap viv things before passing to ZarrPixelSource. Code adapted from feat: Support Zarr v3 (Replace zarr.js with zarrita.js) hms-dbmi/vizarr#172
• Convert from strided to nested in Polygon loader
• Use es2020 as target since es2019 does not support BigInt.
  • Previous reasoning: Use es2019 in vite config #1553
  • Now, portal-ui is on Webpack 5 so it might no longer be an issue to use es2020. (Not sure if other users of vitessce are still on Webpack 4)

TODO

• Use Zarrita in higlass-zarr-datafetchers WIP: zarr dependency higlass/higlass-zarr-datafetchers#2
• Contribute a ZarritaPixelSource to viv: Current ZarrPixelSource does not tree-shake-out ZarrJS because it imports its slice function Allow for passing slice type hms-dbmi/viv#698
• Contribute to Zarrita
  • Docs: How to access .attrs
    • Need to open() after root() in order to access
  • Docs: Example of accessing strided array
  • Docs: Need to await zarr.root
  • Docs: Need to put requestInit in FetchStore(url, { overrides: requestInit });
  • API docs
• Way to prioritize v2 over v3 when opening to avoid extra requests
  • idea: Store can hold counters for how many v2 / v3 metadata files it has "seen" so far. Then can prioritize v2/v3 based on which counter is higher Version counters manzt/zarrita.js#109
• Discuss splitting FileSystemStore into separate package to avoid node dependencies in same package as FetchStore (causes problems in webpack 5 resolve.fallback and resolve.alias do not work with node: prefix webpack/webpack#14166)
• Add github links to package.json files so the links appear on NPM pages https://github.com/manzt/zarrita.js/blob/main/packages/zarrita/package.json
• Add tests / examples which use ES6 Map as an in-memory store
• FileSystemAccess-based Store implementation?
• consolidated metadata function or computed property on root?
• Explore decoding off main thread - Zarr Loader Decoding Off Main Thread hms-dbmi/viv#142 and image load failures due to bad abort support in geotiff hms-dbmi/viv#604 (comment)
• Support for AbortController on async stores More careful check for zarr store that supports abort controller hms-dbmi/viv#299

[github-actions]

Size Change: +874 kB (+8%) 🔍

Total Size: 12.5 MB

Filename	Size	Change
./packages/main/prod/dist/hglib-********.js	4.33 MB	-2.76 kB (0%)
./packages/main/prod/dist/blosc-********.js	448 kB	+448 kB (new file)	🆕
./packages/main/prod/dist/lz4-********.js	51.1 kB	+51.1 kB (new file)	🆕
./packages/main/prod/dist/pako.esm-856454b6-********.js	69 kB	+69 kB (new file)	🆕
./packages/main/prod/dist/zstd-********.js	308 kB	+308 kB (new file)	🆕

ℹ️ View Unchanged

Filename	Size	Change
./packages/main/prod/dist/deflate-********.js	243 B	0 B
./packages/main/prod/dist/index-********.js	7.15 MB	-352 B (0%)
./packages/main/prod/dist/index.min.js	726 B	0 B
./packages/main/prod/dist/jpeg-********.js	15.3 kB	0 B
./packages/main/prod/dist/lerc-********.js	47.2 kB	-18 B (0%)
./packages/main/prod/dist/lzw-********.js	2.1 kB	0 B
./packages/main/prod/dist/packbits-********.js	576 B	0 B
./packages/main/prod/dist/pako.esm-********.js	68.6 kB	0 B
./packages/main/prod/dist/raw-********.js	168 B	0 B
./packages/main/prod/dist/webimage-********.js	836 B	-36 B (-4%)
./packages/main/prod/dist/gzip-********.js	517 B	+517 B (new file)	🆕
./packages/main/prod/dist/zlib-********.js	523 B	+523 B (new file)	🆕

_{compressed-size-action}

[github-actions]

Coverage Report

Status	Category	Percentage	Covered / Total
🔵	Lines	75.58%	10356 / 13701
🔵	Statements	75.58%	10356 / 13701
🔵	Functions	62.6%	288 / 460
🔵	Branches	79.05%	951 / 1203

File Coverage

File	Stmts	% Branch	% Funcs	% Lines	Uncovered Lines
Changed Files
packages/file-types/zarr/src/AnnDataSource.js	80.89%	77.77%	75%	80.89%	23-24, 38-41, 48-49, 51-52, 80-84, 100-104, 145-146, 157-158, 168-177
packages/file-types/zarr/src/ZarrDataSource.js	100%	50%	100%	100%
packages/file-types/zarr/src/anndata-loaders/ObsFeatureMatrixAnndataLoader.js	75.45%	84.21%	73.68%	75.45%	15-27, 49-50, 53, 68-72, 112-113, 138-139, 180-181, 214-215, 249-250, 273-296, 340-350, 353-366, 369-382

Generated in workflow #3220

[manzt]

Cool! Want to chat this week about the zarrita TODOs? I can take a stab at improving these

[keller-mark]

@manzt yes! Will follow up on slack