From ddd7287816aaadc37a660c01d405f9d534e72024 Mon Sep 17 00:00:00 2001 From: jmoore Date: Wed, 14 Jul 2021 10:16:34 +0200 Subject: [PATCH 1/5] Migrate 0.2 changes to latest --- latest/index.bs | 61 +++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 51 insertions(+), 10 deletions(-) diff --git a/latest/index.bs b/latest/index.bs index 7c2a5c7d..82e1a11c 100644 --- a/latest/index.bs +++ b/latest/index.bs @@ -19,7 +19,7 @@ Abstract: This document contains next-generation file format (NGFF) Abstract: specifications for storing bioimaging data in the cloud. Abstract: All specifications are submitted to the https://image.sc community for review. Status Text: The current released version of this specification is -Status Text: 0.1. Migration scripts +Status Text: 0.2. Migration scripts Status Text: will be provided between numbered versions. Data written with these latest changes Status Text: (an "editor's draft") will not necessarily be supported. @@ -127,10 +127,11 @@ multiple levels of resolutions and optionally associated labels. │ ├── .zarray # All image arrays are 5-dimensional │ │ # with dimension order (t, c, z, y, x). │ │ - │ ├── 0.0.0.0.0 # Chunks are stored with the flat directory layout. - │ │ ... # Each dotted component of the chunk file represents - │ └── t.c.z.y.x # a "chunk coordinate", where the maximum coordinate - │ # will be `dimension_size / chunk_size`. + │ └─ t # Chunks are stored with the nested directory layout. + │ └─ c # All but the last chunk element are stored as directories. + │ └─ z # The terminal chunk is a file. Together the directory and file names + │ └─ y # provide the "chunk coordinate" (t, c, z, y, x), where the maximum coordinate + │ └─ x # will be `dimension_size / chunk_size`. │ └── labels │ @@ -207,8 +208,46 @@ keys as specified below for discovering certain types of data, especially images Metadata about the multiple resolution representations of the image can be found under the "multiscales" key in the group-level metadata. -The specification for the multiscale (i.e. "resolution") metadata is provided -in [zarr-specs#50](https://github.com/zarr-developers/zarr-specs/issues/50). + +"multiscales" contains a list of dictionaries where each entry describes a multiscale image. + +Each dictionary contained in the list MUST contain the field "datasets", which is a list of dictionaries describing +the arrays storing the individual resolution levels. +Each dictionary in "datasets" MUST contain the field "path", whose value contains the path to the array for this resolution relative +to the current zarr group. The "path"s MUST be ordered from largest (i.e. highest resolution) to smallest. + +It SHOULD contain the field "name". + +It SHOULD contain the field "version", which indicates the version of the +multiscale metadata of this image (current version is 0.2). + +It SHOULD contain the field "type", which gives the type of downscaling method used to generate the multiscale image pyramid. + +It SHOULD contain the field "metadata", which contains a dictionary with additional information about the downscaling method. + +```json +{ + "multiscales": [ + { + "version": "0.2", + "name": "example", + "datasets": [ + {"path": "0"}, + {"path": "1"}, + {"path": "2"} + ], + "type": "gaussian", + "metadata": { # the fields in metadata depend on the downscaling implementation + "method": "skimage.transform.pyramid_gaussian", # here, the paramters passed to the skimage function are given + "version": "0.16.1", + "args": "[true]", + "kwargs": {"multichannel": true} + } + } + ] +} +``` + If only one multiscale is provided, use it. Otherwise, the user can choose by name, using the first multiscale as a fallback: @@ -223,9 +262,6 @@ if not datasets: datasets = [x["path"] for x in multiscales[0]["datasets"]] ``` -The subresolutions in each multiscale are ordered from highest-resolution -to lowest. - "omero" metadata {#omero-md} ---------------------------- @@ -551,6 +587,11 @@ Version History {#history} Description + + 0.2.0 + 2021-03-29 + Change chunk dimension separator to "/" + 0.1.4 2020-11-26 From c1d92fa8a60063f2ba3aa7d5d63159e078d7b3b5 Mon Sep 17 00:00:00 2001 From: jmoore Date: Wed, 14 Jul 2021 10:24:42 +0200 Subject: [PATCH 2/5] Add bdv-ome-zarr implementation --- 0.2/index.bs | 13 ++++++++----- latest/index.bs | 14 +++++++++----- 2 files changed, 17 insertions(+), 10 deletions(-) diff --git a/0.2/index.bs b/0.2/index.bs index 97948f87..b893c8d2 100644 --- a/0.2/index.bs +++ b/0.2/index.bs @@ -537,11 +537,17 @@ Projects which support reading and/or writing OME-NGFF data include:
+
[bigdataviewer-ome-zarr](https://github.com/mobie/bigdataviewer-ome-zarr)
+
Fiji-plugin for reading OME-Zarr.
+ +
[bioformats2raw](https://github.com/glencoesoftware/bioformats2raw)
+
A performant, Bio-Formats image file format converter.
+
[omero-ms-zarr](https://github.com/ome/omero-ms-zarr)
-
A microservice for OMERO.server that converts images stored in OMERO to OME Zarr files on the fly, served via a web API.
+
A microservice for OMERO.server that converts images stored in OMERO to OME-Zarr files on the fly, served via a web API.
[idr-zarr-tools](https://github.com/IDR/idr-zarr-tools)
-
A full workflow demonstrating the conversion of IDR images to OME Zarr images on S3.
+
A full workflow demonstrating the conversion of IDR images to OME-Zarr images on S3.
[OMERO CLI Zarr plugin](https://github.com/ome/omero-cli-zarr)
An OMERO CLI plugin that converts images stored in OMERO.server into a local Zarr file.
@@ -549,9 +555,6 @@ Projects which support reading and/or writing OME-NGFF data include:
[ome-zarr-py](https://github.com/ome/ome-zarr-py)
A napari plugin for reading ome-zarr files.
-
[bioformats2raw](https://github.com/glencoesoftware/bioformats2raw)
-
A performant, Bio-Formats image file format converter.
-
[vizarr](https://github.com/hms-dbmi/vizarr/)
A minimal, purely client-side program for viewing Zarr-based images with Viv & ImJoy.
diff --git a/latest/index.bs b/latest/index.bs index 82e1a11c..145d3e1f 100644 --- a/latest/index.bs +++ b/latest/index.bs @@ -538,11 +538,17 @@ Projects which support reading and/or writing OME-NGFF data include:
+
[bigdataviewer-ome-zarr](https://github.com/mobie/bigdataviewer-ome-zarr)
+
Fiji-plugin for reading OME-Zarr.
+ +
[bioformats2raw](https://github.com/glencoesoftware/bioformats2raw)
+
A performant, Bio-Formats image file format converter.
+
[omero-ms-zarr](https://github.com/ome/omero-ms-zarr)
-
A microservice for OMERO.server that converts images stored in OMERO to OME Zarr files on the fly, served via a web API.
+
A microservice for OMERO.server that converts images stored in OMERO to OME-Zarr files on the fly, served via a web API.
[idr-zarr-tools](https://github.com/IDR/idr-zarr-tools)
-
A full workflow demonstrating the conversion of IDR images to OME Zarr images on S3.
+
A full workflow demonstrating the conversion of IDR images to OME-Zarr images on S3.
[OMERO CLI Zarr plugin](https://github.com/ome/omero-cli-zarr)
An OMERO CLI plugin that converts images stored in OMERO.server into a local Zarr file.
@@ -550,14 +556,12 @@ Projects which support reading and/or writing OME-NGFF data include:
[ome-zarr-py](https://github.com/ome/ome-zarr-py)
A napari plugin for reading ome-zarr files.
-
[bioformats2raw](https://github.com/glencoesoftware/bioformats2raw)
-
A performant, Bio-Formats image file format converter.
-
[vizarr](https://github.com/hms-dbmi/vizarr/)
A minimal, purely client-side program for viewing Zarr-based images with Viv & ImJoy.
+ Diagram of related projects From 824721117d69917661598752dedce919c29f9398 Mon Sep 17 00:00:00 2001 From: jmoore Date: Wed, 14 Jul 2021 10:32:19 +0200 Subject: [PATCH 3/5] Add links for reviewing to README --- README.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/README.md b/README.md index 3114dfdd..7638188d 100644 --- a/README.md +++ b/README.md @@ -16,6 +16,13 @@ commit to automatically convert to respec/html, via bikeshed. [Learn more about bikeshed](https://w3c-ccg.github.io/bikeshed_instructions.html) +## Reviewing + +Commits on GitHub can be viewed using web services from the W3C: + + * Rendered page: http://api.csswg.org/bikeshed/?url=https://raw.githubusercontent.com/ome/ngff/master/0.2/index.bs + * Diff: https://services.w3.org/htmldiff?doc1=https%3A%2F%2Fngff.openmicroscopy.org%2F0.2%2F&doc2=http%3A%2F%2Fapi.csswg.org%2Fbikeshed%2F%3Furl%3Dhttps%3A%2F%2Fraw.githubusercontent.com%2Fome%2Fngff%2Fmaster%2Flatest%2Findex.bs + ## Citing Please see https://ngff.openmicroscopy.org/latest#citing for the latest From 5a3b5a3447f4fa7afe7795da3973fddbce6775bb Mon Sep 17 00:00:00 2001 From: jmoore Date: Tue, 3 Aug 2021 11:59:38 +0200 Subject: [PATCH 4/5] Replace 0.1 with 0.2 --- 0.2/index.bs | 2 +- latest/index.bs | 14 +++++++------- 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/0.2/index.bs b/0.2/index.bs index b893c8d2..6c731c6d 100644 --- a/0.2/index.bs +++ b/0.2/index.bs @@ -573,7 +573,7 @@ Citing {#citing} ================ [Next-generation file format (NGFF) specifications for storing bioimaging data in the cloud.](https://ngff.openmicroscopy.org/0.2) -J. Moore, *et al*. Editors. Open Microscopy Environment Consortium, 20 November 2020. +J. Moore, *et al*. Editors. Open Microscopy Environment Consortium, 29 March, 2021. This edition of the specification is [https://ngff.openmicroscopy.org/0.2/](https://ngff.openmicroscopy.org/0.2/]). The latest edition is available at [https://ngff.openmicroscopy.org/latest/](https://ngff.openmicroscopy.org/latest/). [(doi:10.5281/zenodo.4282107)](https://doi.org/10.5281/zenodo.4282107) diff --git a/latest/index.bs b/latest/index.bs index 145d3e1f..2b598a4a 100644 --- a/latest/index.bs +++ b/latest/index.bs @@ -271,7 +271,7 @@ can be found under the "omero" key in the group-level metadata: ```json "id": 1, # ID in OMERO "name": "example.tif", # Name as shown in the UI -"version": "0.1", # Current version +"version": "0.2", # Current version "channels": [ # Array matching the c dimension size { "active": true, @@ -348,7 +348,7 @@ above). ```json "image-label": { - "version": "0.1", + "version": "0.2", "colors": [ { "label-value": 1, @@ -460,7 +460,7 @@ For example the following JSON object defines a plate with two acquisition and "name": "B" } ], - "version": "0.1", + "version": "0.2", "wells": [ { "path": "2020-10-10/A/1" @@ -527,7 +527,7 @@ the last two fields of view were part of the second acquisition. "path": "3" } ], - "version": "0.1" + "version": "0.2" } ``` @@ -574,9 +574,9 @@ Note: If you would like to see your project listed, please open an issue or PR o Citing {#citing} ================ -[Next-generation file format (NGFF) specifications for storing bioimaging data in the cloud.](https://ngff.openmicroscopy.org/0.1) -J. Moore, *et al*. Editors. Open Microscopy Environment Consortium, 20 November 2020. -This edition of the specification is [https://ngff.openmicroscopy.org/0.1/](https://ngff.openmicroscopy.org/0.1/]). +[Next-generation file format (NGFF) specifications for storing bioimaging data in the cloud.](https://ngff.openmicroscopy.org/0.2) +J. Moore, *et al*. Editors. Open Microscopy Environment Consortium, 29 March, 2021. +This edition of the specification is [https://ngff.openmicroscopy.org/0.2/](https://ngff.openmicroscopy.org/0.2/]). The latest edition is available at [https://ngff.openmicroscopy.org/latest/](https://ngff.openmicroscopy.org/latest/). [(doi:10.5281/zenodo.4282107)](https://doi.org/10.5281/zenodo.4282107) From d288615965bf7ea79d1096d57fe0d377881d2069 Mon Sep 17 00:00:00 2001 From: jmoore Date: Tue, 3 Aug 2021 12:01:07 +0200 Subject: [PATCH 5/5] Fix whitespace differences --- 0.2/index.bs | 1 - latest/index.bs | 1 - 2 files changed, 2 deletions(-) diff --git a/0.2/index.bs b/0.2/index.bs index 6c731c6d..4e25ad2c 100644 --- a/0.2/index.bs +++ b/0.2/index.bs @@ -22,7 +22,6 @@ Status Text: will be provided between numbered versions. Data written with the l Status Text: (an "editor's draft") will not necessarily be supported. - Introduction {#intro} ===================== diff --git a/latest/index.bs b/latest/index.bs index 2b598a4a..fbe6bed4 100644 --- a/latest/index.bs +++ b/latest/index.bs @@ -561,7 +561,6 @@ Projects which support reading and/or writing OME-NGFF data include:
- Diagram of related projects