release_notes_v21.10.00

Version 21.10.00 (Oct 06, 2021)

This version would be available through both Conda (https://anaconda.org/rapidsai/cucim) and PyPI package (https://pypi.org/project/cucim/21.10.0/).

🚀 New Features

1. Add transforms for Digital Pathology (#100) @shekhardw @chirayuG-nvidia
2. Enable GDS and Support Runtime Context (__enter__, __exit__) for CuFileDriver and CuImage (#106) @gigony
3. Add a mechanism for user to know the availability of cucim.CuImage (#107) @gigony
4. Support raw RGB tiled TIFF (#108) @gigony

1. Add high-performance transforms for Digital Pathology (#100)

We have implemented the following methods in this release.

• cucim.core.operations.color module
  • color_jitter
• cucim.core.operations.intensity module
  • rand_zoom
  • scale_intensity_range
  • zoom
• cucim.core.operations.spatial module
  • image_flip
  • image_rotate_90
  • rand_image_flip
  • rand_image_rotate_90

The above methods are exposed to MONAI (CuCIM Transforms #2932) through cucim.core.operations.expose.transform module

• color_jitter
• image_flip
• image_rotate_90
• scale_intensity_range
• zoom
• rand_zoom
• rand_image_flip
• rand_image_rotate_90

Special thanks to Chirayu Garg(@chirayuG-nvidia) for implementing C++ CUDA Kernels of the above transform methods!

2. Enable GDS and Support Runtime Context (__enter__, __exit__) for CuFileDriver and CuImage (#106)

Enabling GDS

GPU-Direct-Storage feature was disabled before (#3) because there was no public package available.

Now we re-enabled the feature. It's up to the user to install GDS libraries (GDS libraries are available through CUDA Toolkit >=11.x), or can install manually by following the instruction https://docs.nvidia.com/gpudirect-storage/troubleshooting-guide/index.html#gds-installing

If the GDS library is not available, the filesystem API still works with compatibility mode (our own implementation to do the same behavior).

Please see the Tutorial (Accessing File with GDS) for detail.

Support Runtime Context for CuFileDriver

CuFile Driver now can use Runtime Context so with statement is available.

with for cucim.clara.filesystem.open() would be available in the next release (v21.12.00)

from cucim.clara.filesystem import CuFileDriver
import cucim.clara.filesystem as fs
import os, cupy as cp, torch


# Assume a file ('nvme/input.raw') with size 10 in bytes

# Create a CuPy array with size 10 (in bytes)
cp_arr = cp.ones(10, dtype=cp.uint8)
# Create a PyTorch array with size 10 (in bytes)
cuda0 = torch.device('cuda:0')
torch_arr = torch.ones(10, dtype=torch.uint8, device=cuda0)

print(f"fs.is_gds_available(): {fs.is_gds_available()}")

# Using CuFileDriver
# (Opening a file with O_DIRECT flag is required for GDS)
fno = os.open("nvme/input.raw", os.O_RDONLY | os.O_DIRECT)
with CuFileDriver(fno) as fd:
    # Read 8 bytes starting from file offset 0 into buffer offset 2
    read_count = fd.pread(cp_arr, 8, 0, 2)
    # Read 10 bytes starting from file offset 3
    read_count = fd.pread(torch_arr, 10, 3)

# Another way of opening file with cuFile (>= v21.12.00)
# with fs.open("nvme/output.raw", "w") as fd:
#     # Write 10 bytes from cp_array to file starting from offset 5
#     write_count = fd.pwrite(cp_arr, 10, 5)
fd = fs.open("nvme/output.raw", "w")
# Write 10 bytes from cp_array to file starting from offset 5
write_count = fd.pwrite(cp_arr, 10, 5)
fd.close()

Support Runtime Context for CuImage

CuImage class didn't have close() method (file handle is close when the object is destructed) but now close() method is available.

Once the file handle for the CuImage object is closed, it cannot call operations that require file access.

from cucim import CuImage

with CuImage("image.tif") as cuimg:
  region = cuimg.read_region((0, 0), (100, 100))

print(region.shape)
region2 = cuimg.read_region((100, 100), (200, 200))

[Plugin: cucim.kit.cuslide] Loading...
[Plugin: cucim.kit.cuslide] Loading the dynamic library from: /ssd/repo/cucim/python/cucim/src/cucim/clara/cucim.kit.cuslide@21.10.00.so
[Plugin: cucim.kit.cuslide] loaded successfully. Version: 0
Initializing plugin: cucim.kit.cuslide (interfaces: [cucim::io::IImageFormat v0.1]) (impl: cucim.kit.cuslide)
[100, 100, 3]
Traceback (most recent call last):
  File "test_cucim.py", line 7, in <module>
    region2 = cuimg.read_region((100, 100), (200, 200))
RuntimeError: [Error] The image file is closed!
[Plugin: cucim.kit.cuslide] Unloaded.

3. Add a mechanism for users to know the availability of cucim.CuImage (#107)

Added cucim.is_available(module_name) method. (module_name (str): Name of the module to check. (e.g. "skimage", "core", and "clara"))

Users can query if a specific module is available or not.

For example, if cucim.clara module which requires C shared library is not available:

>>> import cucim
>>> cucim.is_available()
False
>>> cucim.is_available("skimage")
True
>>> cucim.is_available("core")
True
>>> cucim.is_available("clara")
False

4. Support raw RGB tiled TIFF (#108)

Due to #17 ([BUG] Compression scheme 33003 tile decoding is not implemented), #19 (Check compression method used in the image) was merged so cuCIM has been handling only particular formats (only jpeg/deflate-compressed image).

If the input TIFF image has no-compressed RAW tile image, cuCIM showed the following error message:

RuntimeError: This format (compression: 1, sample_per_pixel: 1, planar_config: 1, photometric: 1) is not supported yet!.

https://github.com/Project-MONAI/MONAI/pull/2987/checks?check_run_id=3667530814#step:7:15641

Now cuCIM supports a raw RGB tiled TIFF with fast-path, re-allowing non-compressed image with slow-path (in case the input is not tiled RGB image).

This supports https://github.com/Project-MONAI/MONAI/pull/2987.

🐛 Bug Fixes

• fix failing regionprops test cases (#110) @grlee77

📖 Documentation

• Forward-merge branch-21.08 to branch-21.10 (#88) @jakirkham
• Update PyPI cuCIM v21.08.01 README.md and CHANGELOG.md (#87) @gigony

🛠️ Improvements

• ENH Replace gpuci_conda_retry with gpuci_mamba_retry (#69) @dillon-cullinan