Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Proxy store for nested chunks #85

Merged
merged 6 commits into from
Apr 20, 2021
Merged

Use Proxy store for nested chunks #85

Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
01a051a
Use Proxy store for nested chunks
will-moore Mar 30, 2021
ddee2e9
Only use nested(store) for correct version of OME-Zarr
will-moore Mar 31, 2021
c2bbed8
Merge remote-tracking branch 'origin/master' into nested_chunks
will-moore Mar 31, 2021
e10b2ae
format fixes
will-moore Mar 31, 2021
3cdabea
Move nested handling to io.ts
will-moore Mar 31, 2021
b879cdb
Revert changes to ome.ts, removing isNested duplication
will-moore Apr 8, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 22 additions & 2 deletions src/utils.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,29 @@ async function normalizeStore(source: string | ZarrArray['store']) {

export async function open(source: string | ZarrArray['store']) {
const { store, path } = await normalizeStore(source);
return openGroup(store, path).catch((err) => {

function nested(store) {
const get = (target, key) => {
will-moore marked this conversation as resolved.
Show resolved Hide resolved
if (key === 'getItem' || key === 'setItem' || key === 'containsItem') {
return (path, ...args) => {
if (path.endsWith('.zarray') || path.endsWith('.zattrs') || path.endsWith('.zgroup')) {
return target[key](path, ...args);
}
const prefix = path.split('/');
const chunkKey = prefix.pop();
const newPath = [...prefix, chunkKey.replaceAll('.', '/')].join('/');
return target[key](newPath, ...args);
}
}
return Reflect.get(target, key);
};
return new Proxy(store, { get });
}
const nested_store = nested(store);
Copy link
Member

@manzt manzt Mar 30, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would actually move this to be a utility function (outside of open), since this would break vizarr for folks who don't have a nested store (e.g. imjoy usage, Jupyter notebooks).

My thought is that in io.ts, we can check for OME-Zarr version and replace the underlying store with a proxy if needed.

if (node instanceof ZarrGroup) {
    const attrs = (await node.attrs.asObject()) as Ome.Attrs;

  // not sure where the v0.2 meta data is located, but something similar to this.
   if (attrs.version === "0.2") {
       node.store = nested(node.store); // replaces underlying store with proxied store.
    }

    if ('plate' in attrs) {
      return loadPlate(config, node, attrs.plate);
    }

    if ('well' in attrs) {
      return loadWell(config, node, attrs.well);
    }

/* ... */


return openGroup(nested_store, path).catch((err) => {
if (err instanceof ContainsArrayError) {
return openArray({ store, path });
return openArray({ nested_store, path });
}
throw err;
});
Expand Down