Allow creating spatial pyramid levels

[forman]

xcube-server is encountering massive performance problems when low-res tiles are created from spatial hi-res dataset, especially when their chunking is not ideal for tile extraction in the spatial dimensions. This is because spatial resolution levels are computed on the fly. For the lowest resolution (level zero) tiles, all hi-res data need to be read.

We need a data format that allows xcube-server to read from spatial pyramid levels, if they exist, and a tool that can generate spatial pyramid levels from hi-res datasets.

Format suggestion

Let some/file/path/bigdata.zarr be the path to an hi-res dataset, then physical representation of the spatial pyramid with 8 levels could be as follows:

    - some/file/path/bigdata.zarr
    - some/file/path/bigdata.levels/
      - 0.lnk        # contains link to original dataset at spatial resolution res0
      - 1.zarr/     # First downsampled level with res = res0 * 2^1
      - 2.zarr/     # Second downsampled level with res = res0 * 2^2
      ...
      - 7.zarr/     # Second downsampled level with res = res0 * 2^7

All levels have the same chunking. The number of chunks in one of the spatial dimensions at highest level is one, all other levels have multiple chunks in spatial dimensions.

Another possibility is that all the levels go into a single ZARR dataset.

[forman]

Done!