examples/tile_stitching.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "38bdfeb0-8389-4d8e-8cba-5b6aa2080df6",
   "metadata": {},
   "source": [
    "# Preprocessing: Tile Stitching\n",
    "\n",
    "\n",
    "[![View on GitHub](https://img.shields.io/badge/View-on%20GitHub-lightgrey?logo=github)](https://github.com/Dana-Farber-AIOS/pathml/blob/master/examples/)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "109a91e1-569b-4766-a157-d7921d37280e",
   "metadata": {},
   "source": [
    "In the rapidly evolving field of digital pathology, handling and processing high-resolution histopathology images is a critical task. This is especially true in the context of whole-slide imaging (WSI), a technique that has revolutionized the analysis of tissue samples by digitizing entire microscope slides at a gigapixel scale. However, the large size of these images presents a significant challenge in terms of data management and analysis. To address this, the images are often segmented into smaller, manageable, and overlapping segments known as tiles. The real challenge, and the focus of this tutorial, is in the accurate reconstruction of these tiles to reform the complete image—a process known as tile stitching.\n",
    "\n",
    "PathML offers the TileStitcher class, which is the Python adaptation of an existing Groovy script used in QuPath which is available [here](https://gist.github.com/petebankhead/b5a86caa333de1fdcff6bdee72a20abe). The TileStitcher class reimplements the functionality of its Groovy counterpart, allowing for the extraction of tile coordinates from the baseline TIFF tags followed by seamlessly stitching them and writing the stitched image as a pyramidal OME-TIFF file.\n",
    "\n",
    "This tutorial will walk you through the process of using TileStitcher class to collect, parse, and stitch large sets of tiled TIFF images then saving the reconstructed image."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c6a876e-95c8-4b62-80dc-2de356f50daf",
   "metadata": {},
   "source": [
    "\n",
    "## Prerequisites\n",
    "\n",
    "Before using the `TileStitcher` class, we need to install the necessary software and configure the environment.\n",
    "\n",
    "### Software Installation\n",
    "\n",
    "The `TileStitcher` class requires QuPath and OpenJDK. Here is how to install them:\n",
    "\n",
    "1. Download and install QuPath from its [GitHub release page](https://github.com/qupath/qupath/releases). Here we are using version 0.3.1.\n",
    "\n",
    "```bash\n",
    "wget https://github.com/qupath/qupath/releases/download/v0.4.3/QuPath-0.4.3-Linux.tar.xz\n",
    "```\n",
    "Unzip\n",
    "```bash\n",
    "   tar -xvf QuPath-0.4.3-Linux.tar.xz\n",
    "```\n",
    "\n",
    "Make executable\n",
    "\n",
    "```bash\n",
    "chmod u+x /path/to/QuPath/bin/QuPath\n",
    "```\n",
    "2. Download and Install OpenJDK 17\n",
    "\n",
    "```bash\n",
    "wget https://download.oracle.com/java/17/latest/jdk-17_linux-x64_bin.deb\n",
    "sudo apt install ./jdk-17_linux-x64_bin.deb\n",
    "\n",
    "   Set the Java path according to your installation method. If you have set up your environment using PathML, set the Java path to `/opt/conda/envs/pathml`. Otherwise, adjust it to the appropriate path on your system.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b45ddfe-636e-4998-add8-52d16db7bc1c",
   "metadata": {},
   "source": [
    "### Environment Configuration"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a29c5e29-e7fe-497f-8f79-0d9de6ff2469",
   "metadata": {},
   "source": [
    "To use `TileStitcher`, we need to set the correct paths to the QuPath library and OpenJDK. For this, we need to add the paths to the environment variables `JAVA_HOME`, `CLASSPATH`, and `LD_LIBRARY_PATH`.\n",
    "\n",
    "The `JAVA_HOME` environment variable should be set to the path where the JDK is installed.\n",
    "The `CLASSPATH` environment variable should include paths to all the QuPath library jar files.\n",
    "In the initialization of TileStitcher, these environment variables are used to start the Java Virtual Machine (JVM) and import the necessary QuPath classes."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3be9706-2c25-43e1-b98f-24ac84f101fb",
   "metadata": {},
   "source": [
    "## Best Practices and Considerations for Using the TileStitcher Module\n",
    "\n",
    "### 1. JVM Session Management\n",
    "\n",
    "The TileStitcher module utilizes jpype to manage the JVM sessions, a departure from the python-javabridge used in other parts of the package. This difference can cause conflicts when trying to run modules concurrently within the same Python environment. Hence, it is advisable to avoid running TileStitcher operations in the same notebook where python-javabridge dependent modules are running.\n",
    "\n",
    "### 2. Restarting Kernel to Re-initialize JVM\n",
    "\n",
    "The jpype does not allow the JVM to be restarted within the same Python session once it has been terminated. As a result, to run the TileStitcher module again or to switch back to modules that use python-javabridge, a kernel restart might be necessary.\n",
    "\n",
    "### 3. Segregating Workflows\n",
    "\n",
    "To mitigate potential conflicts, consider segregating workflows based on the JVM management package they depend on. It is recommended to use separate notebooks or scripts for operations involving TileStitcher and for those involving modules that are dependent on python-javabridge.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bc1f38e6-1119-427d-9a86-a1d9461600b2",
   "metadata": {},
   "source": [
    "### Using TileStitcher\n",
    "\n",
    "Ensure QuPath and JDK installations are complete before proceeding.\n",
    "\n",
    "#### Initialization\n",
    "\n",
    "The class is initialized with several parameters:\n",
    "\n",
    "- `qupath_jarpath`: List of paths to QuPath JAR files.\n",
    "\n",
    "- `java_path`: Custom path to Java installation. If set, `JAVA_HOME` will be overridden.\n",
    "\n",
    "- `memory`: Allocated memory for the JVM (default: \"40g\").\n",
    "\n",
    "- `bfconvert_dir`: Directory for Bio-Formats conversion tools.\n",
    "\n",
    "During initialization, `TileStitcher` sets up the Java Virtual Machine (JVM) and imports necessary QuPath classes. It also includes error handling for Java path configurations and JVM startup issues.\n",
    "\n",
    "#### JVM Startup\n",
    "\n",
    "The `_start_jvm` method initiates the JVM with specified memory and classpath settings. It imports necessary QuPath classes upon successful startup, ensuring compatibility with Java 17."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "76ecba97-d4b9-4ee7-856f-3e8ca267dcf5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "./tools/bftools/bfconvert ./tools/bftools/bf.sh\n",
      "bfconvert version: Version: 7.0.1\n",
      "Build date: 16 October 2023\n",
      "VCS revision: 20e58cef1802770cc56ecaf1ef6f323680e4cf65\n",
      "Setting Environment Paths\n",
      "Java Home is already set\n",
      "JVM was already started\n"
     ]
    }
   ],
   "source": [
    "import glob\n",
    "import os\n",
    "from pathml.preprocessing.tilestitcher import TileStitcher\n",
    "from pathml.utils import setup_qupath\n",
    "\n",
    "\n",
    "# Set the path to the JDK\n",
    "os.environ[\"JAVA_HOME\"] = \"/usr/lib/jvm/jdk-17/\"\n",
    "\n",
    "# Use setup_qupath to get the QuPath installation path\n",
    "qupath_home = setup_qupath(\"../../tools1/tools1/\")\n",
    "\n",
    "if qupath_home is not None:\n",
    "    os.environ[\"QUPATH_HOME\"] = qupath_home\n",
    "\n",
    "    # Construct the path to QuPath jars based on qupath_home\n",
    "    qupath_jars_dir = os.path.join(qupath_home, \"lib\", \"app\")\n",
    "    qupath_jars = glob.glob(os.path.join(qupath_jars_dir, \"*.jar\"))\n",
    "    qupath_jars.append(os.path.join(qupath_jars_dir, \"libopenslide-jni.so\"))\n",
    "\n",
    "    # Create an instance of TileStitcher\n",
    "    stitcher = TileStitcher(qupath_jars)\n",
    "else:\n",
    "    print(\"QuPath installation not found. Please check the installation path.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "9a817b38-4d28-4d62-a365-6a2fe584b455",
   "metadata": {},
   "outputs": [],
   "source": [
    "import jpype"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "27d6403a-a249-4a4b-9a66-29f9494fd6f7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "jpype.isJVMStarted()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "77df2040-ca5c-41e8-b9d2-64ad2a0248e9",
   "metadata": {},
   "source": [
    "### Image Stitching with TileStitcher\n",
    "\n",
    "Once `TileStitcher` is initialized, it's capable of stitching together tiled images.\n",
    "\n",
    "-   Method: `run_image_stitching`\n",
    "-   Inputs:\n",
    "    -   A list of TIFF files or a directory containing TIFF files.\n",
    "    -   Output file path.\n",
    "-   Optional Parameters:\n",
    "    -   `downsamples`: Specify the number of downsample levels (e.g., `[1,4,8]`). Defaults to levels read from the tiles.\n",
    "    -   `separate_series`: If set to `True`, it downloads bftools and extracts the base level image from the stitched image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "0838c482-ce15-4f5c-8d05-24c68b9d22be",
   "metadata": {},
   "outputs": [],
   "source": [
    "input_files = glob.glob(\"path/to/tiles/*.tif\")\n",
    "output_file = \"path/to/output.ome.tif\"\n",
    "stitcher.run_image_stitching(input_files, output_file)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2d3d6536-7c7f-42f0-b49b-9066b84489c0",
   "metadata": {},
   "source": [
    "### **Demo**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "41e3bf35-b771-4ce6-9e3e-6a74d514213b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import jpype"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "841fe169-fd0e-4dce-8551-77f95b1b2a26",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(False, (0, 0, 0))"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "jpype.isJVMStarted(), jpype.getJVMVersion()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "6979bfb0-4b7d-48e0-8714-116055076972",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import glob\n",
    "import os\n",
    "\n",
    "# Set the path to the JDK\n",
    "os.environ[\"JAVA_HOME\"] = \"/opt/conda/envs/pathml\"\n",
    "os.environ[\"PATH\"] += os.pathsep + os.path.join(\"/opt/conda/envs/pathml\", \"bin\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "edb49b7b-651b-4e14-be86-1d74cbc40b82",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using JVM version: (17, 0, 10) from /opt/conda/envs/pathml/lib/jvm/lib/server/libjvm.so\n",
      "Importing required QuPath classes\n"
     ]
    }
   ],
   "source": [
    "from pathml.preprocessing.tilestitcher import TileStitcher\n",
    "from pathml.utils import setup_qupath\n",
    "\n",
    "\n",
    "# Use setup_qupath to get the QuPath installation path\n",
    "qupath_home = setup_qupath(\"./tools/\")\n",
    "\n",
    "if qupath_home is not None:\n",
    "    os.environ[\"QUPATH_HOME\"] = qupath_home\n",
    "\n",
    "    # Construct the path to QuPath jars based on qupath_home\n",
    "    qupath_jars_dir = os.path.join(qupath_home, \"lib\", \"app\")\n",
    "    qupath_jars = glob.glob(os.path.join(qupath_jars_dir, \"*.jar\"))\n",
    "    qupath_jars.append(os.path.join(qupath_jars_dir, \"libopenslide-jni.so\"))\n",
    "\n",
    "    # Create an instance of TileStitcher\n",
    "    stitcher = TileStitcher(qupath_jars)\n",
    "else:\n",
    "    print(\"QuPath installation not found. Please check the installation path.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "b71697c2-84d9-4c98-b0da-8df419a87ad5",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(True, (17, 0, 10))"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "jpype.isJVMStarted(), jpype.getJVMVersion()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "3a627854-7111-41d6-95e9-efb5b6c1d63c",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Specify the folder path where the list of .tif files are present, here we are using a folder path that has single tif file for demo purposes.\n",
    "infile_path = \"../tests/testdata/tilestitching_testdata/\"\n",
    "outfile_path = \"./output/tile_stitching_demo.ome.tif\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "f9e62089-4f7c-4c43-a5d3-70b8b633b207",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "19:07:21.270 [main] [INFO ] q.l.i.s.b.BioFormatsServerOptions - Setting max Bio-Formats readers to 8\n",
      "19:07:21.900 [main] [ERROR] q.l.i.s.o.OpenslideServerBuilder - Could not load OpenSlide native libraries\n",
      "java.lang.UnsatisfiedLinkError: no openslide-jni in java.library.path: /opt/conda/envs/pathml/lib/python3.9/site-packages/cv2/../../lib64:/usr/local/cuda/lib64:/usr/local/nccl2/lib:/usr/local/cuda/extras/CUPTI/lib64:/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib\n",
      "\tat java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2434)\n",
      "\tat java.base/java.lang.Runtime.loadLibrary0(Runtime.java:818)\n",
      "\tat java.base/java.lang.System.loadLibrary(System.java:1993)\n",
      "\tat org.openslide.OpenSlideJNI.<clinit>(OpenSlideJNI.java:55)\n",
      "\tat org.openslide.OpenSlide.<clinit>(OpenSlide.java:53)\n",
      "\tat qupath.lib.images.servers.openslide.OpenslideServerBuilder.<clinit>(OpenslideServerBuilder.java:90)\n",
      "\tat java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)\n",
      "\tat java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)\n",
      "\tat java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)\n",
      "\tat java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)\n",
      "\tat java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)\n",
      "\tat java.base/java.util.ServiceLoader$ProviderImpl.newInstance(ServiceLoader.java:789)\n",
      "\tat java.base/java.util.ServiceLoader$ProviderImpl.get(ServiceLoader.java:729)\n",
      "\tat java.base/java.util.ServiceLoader$3.next(ServiceLoader.java:1403)\n",
      "\tat qupath.lib.images.servers.ImageServerProvider.getServerBuilders(ImageServerProvider.java:191)\n",
      "\tat qupath.lib.images.servers.ImageServerProvider.getPreferredUriImageSupport(ImageServerProvider.java:223)\n",
      "19:07:21.901 [main] [INFO ] q.l.i.s.o.OpenslideServerBuilder - If you want to use OpenSlide, you'll need to get the native libraries (either building from source or with a packager manager)\n",
      "and add them to your system PATH, including openslide-jni.\n",
      "19:07:24.717 [main] [WARN ] q.l.i.writers.ome.OMEPyramidWriter - Deleting existing file /home/jupyter/sreekar/projects/tilestitching/pathml/examples/./output/tile_stitching_demo.ome.tif\n",
      "19:07:24.733 [main] [INFO ] q.l.i.writers.ome.OMEPyramidWriter - Writing Sparse image (1 regions) to /home/jupyter/sreekar/projects/tilestitching/pathml/examples/./output/tile_stitching_demo.ome.tif (series 1/1)\n",
      "19:07:24.734 [main] [INFO ] q.l.i.writers.ome.OMEPyramidWriter - Setting series 0 compression to zlib\n",
      "19:07:24.734 [main] [INFO ] q.l.i.writers.ome.OMEPyramidWriter - Writing resolution 1 of 1 (downsample=1.0, 12 tiles)\n",
      "19:07:24.736 [main] [INFO ] q.l.i.writers.ome.OMEPyramidWriter - Writing plane 1/1\n",
      "19:07:35.528 [main] [INFO ] q.l.i.writers.ome.OMEPyramidWriter - Plane written in 10792 ms\n",
      "Image stitching completed. Output file: ./output/tile_stitching_demo.ome.tif\n",
      "bfconvert version: Version: 7.1.0\n",
      "Build date: 11 December 2023\n",
      "VCS revision: 05c7b2413cfad19a73b619c61ddf77ca2d038ce7\n",
      "./output/tile_stitching_demo.ome.tif\n",
      "OMETiffReader initializing ./output/tile_stitching_demo.ome.tif\n",
      "[OME-TIFF] -> ./output/tile_stitching_demo_separated.ome.tif [OME-TIFF]\n",
      "Reading IFDs\n",
      "Populating metadata\n",
      "Reading IFDs\n",
      "Populating metadata\n",
      "\tConverted 1/7 planes (14%)\n",
      "\tConverted 7/7 planes (100%)\n",
      "Overwriting existing Creator attribute: OME Bio-Formats 6.12.0\n",
      "[done]\n",
      "2.023s elapsed (162.28572+47.857143ms per plane, 507ms overhead)\n",
      "bfconvert completed. Output file: ./output/tile_stitching_demo_separated.ome.tif\n",
      "Original stitched image deleted: ./output/tile_stitching_demo.ome.tif\n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "\n",
    "start = time.time()\n",
    "# Run the image stitching process\n",
    "stitcher.run_image_stitching(\n",
    "    infile_path, outfile_path, downsamples=[1], separate_series=True\n",
    ")\n",
    "end = time.time()"
   ]
  }
 ],
 "metadata": {
  "environment": {
   "kernel": "pathml",
   "name": "pytorch-gpu.1-13.m105",
   "type": "gcloud",
   "uri": "gcr.io/deeplearning-platform-release/pytorch-gpu.1-13:m105"
  },
  "kernelspec": {
   "display_name": "pathml",
   "language": "python",
   "name": "pathml"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}