The Royer Lab at the Chan Zuckerberg Biohub San Francisco is a pluridisciplinary team of computer scientists, optical engineers, and biologists. Our goal is to build a time-resolved, multidimensional atlas of vertebrate development using zebrafish as a model organism. To attain this goal, we design, build, and implement novel state-of-the-art light-sheet microscopes, as well as deep learning–based image processing and analysis algorithms.

This Github org is a satelite of the main CZ Biohub repository and is home to repositories actively developped by members of the Royer Lab on all these topics. Members of the Royer lab also maintain some projects in the main CZB SF org.

• napari-chatgpt(Omega) is a napari plugin that utilizes OpenAI's Large Language Model (LLM) called ChatGPT. It introduces an agent named Omega, which is aware of napari and capable of engaging in conversational image processing and analysis tasks. Omega utilizes the ChatGPT API through the LangChain Python library and napari. Omega demonstrates the potential of LLMs by performing tasks such as writing image processing code, creating napari widgets, correcting coding mistakes, conducting follow-up analysis, and controlling the napari viewer.

• ultrack is a cell-tracking Python package. It's one of the few packages that does joint segmentation and cell tracking using multiple segmentation hypotheses. It leverages hierarchical segmentation and solves the tracking problem as an integer linear program (ILP) to optimize cell segmentation and lineage reconstruction jointly. This allows ultrack to be effective at tracking cells across large microscopy datasets containing millions of segmentation instances in terabyte-scale 3D+t datasets. Furthermore, it achieves competitive results with or without deep learning, bypassing the requirement for scarce 3D annotated data in the fluorescence microscopy field. The package provides utilities for computing the hierarchical segmentations from various inputs, optimizing the ILP tracking formulation, and exporting the results. It is designed to be computationally efficient and support out-of-memory processing of large datasets. You can read the preprint here: Large-Scale Multi-Hypotheses Cell Tracking Using Ultrametric Contours Maps.

• napari-segment-anything is a napari plugin is a napari plugin that integrates Meta's powerful segment anything model (SAM) into the napari multi-dimensional image viewer. SAM is a promptable segmentation model that can generate high-quality masks for objects in an image given simple prompts like points or boxes. The napari-segment-anything plugin provides an interactive graphical user interface that allows users to easily select points or regions in an image and obtain the corresponding segmentation masks from SAM in real-time. This enables very rapid and user-friendly segmentation of objects in napari without needing any specialized training data. The plugin aims to make the most of both SAM's impressive few-shot segmentation capabilities and napari's rich interactive visualization and analysis environment.In other words, for non-napari users, a native Qt interface to SAM.

• DaXi - High-Resolution, Large Imaging Volume, and Multi-View Single Objective Light-Sheet Microscope. The DaXi microscope (Yang et al.) is a novel single-objective light-sheet microscope design that provides high-resolution and larger volume imaging capabilities suitable for imaging living samples such as developing embryos. It offers a wider field of view, multi-view imaging, and higher throughput multi-well imaging via remote coverslip placement. The microscope achieves a resolution of 450 nm laterally and 2 μm axially over an imaging volume of 3,000 × 800 × 300 μm. The microscope has been successfully used to image various systems, including Drosophila egg chamber development, zebrafish whole-brain activity, and zebrafish embryonic development, up to nine embryos at a time. This repository has blueprints and source code.

• Aydin is a tool for image denoising that offers various algorithms and features such as self-supervised, auto-tuned, and unsupervised denoising, and can handle n-dimensional array-structured images with an arbitrary number of dimensions. It comes with a graphical user interface, command line interface, and API for custom coding and integration into scripts and applications. It also has a simplified napari plugin in development, and commercial use is allowed if pyside is used as the GUI backend.

• DEXP is a Python library designed for managing, processing, and visualizing light-sheet microscopy datasets. It is built on top of Napari, CuPy, Zarr, and DASK, and offers various specialized image processing functions, visualization functions, and dataset management functions. DEXP provides GPU acceleration via CuPy and has a fallback option for testing on small datasets. It also has a command line interface that allows non-coders to pipeline large processing jobs from raw data to fully rendered videos in MP4 format.

• Cytoself is a self-supervised platform for learning features of protein subcellular localization from microscopy images (Kobayashi, et al). The representations derived from cytoself encapsulate highly specific features that can derive functional insights for proteins on the sole basis of their localization. Applying cytoself to images of endogenously labeled proteins from the recently released OpenCell database creates a highly resolved protein localization atlas (Cho, et al.).

• ZAF (Zebrafish Automatic Feeder) is an open-source fully automatic daily feeding system for zebrafish, which provides a standardized amount of food to every tank, is cost-efficient and easy to build (Lange et al.). The advanced version, ZAF+, allows for the precise control of food distribution as a function of fish density per tank. The design is modular and can be scaled depending on user needs, and the system does not adversely affect zebrafish culture. Instructions for building both systems, from hardware and user interface to open-source python code, are provided.

• The napari organization and repos were initially created by Loic when he started the project with Juan Nunez Iglesias. Loic is not currently an active maintainer of this project, napari has taken a bright life of its own and is maintained by a vibrant and talented team of core developpers, the best possible outcome.

• Noise2Self is a self-supervised denoising method that does not require ground truth data. It is based on the idea that the same image corrupted by independent noise realizations can provide a training signal for a denoising neural network (Batson & Royer, accepted at ICML 2019). Noise2Self provides high-quality denoising performance on various tasks, such as fluorescence microscopy datasets. The package includes a basic PyTorch implementation. Here is a presentation by Josh (Batson) on Noise2Self. Noise2Self and Noise2Void by the Jug Lab are essentially the same idea discovered independently. The main difference is that we generalized the concept to any multi-dimensional measurements, as exemplified in our BioRxiv preprint Molecular Cross-Validation for Single-Cell RNA-seq. The ideas behind Noise2Self were further extended into a convenient and easy-to-use software package Aydin.

The Chan Zuckerberg Biohub San Francisco is a nonprofit research institute that aims to understand the mechanisms of diseases and develop new technologies to create effective therapies. The institute brings together the leading academic institutions in the San Francisco Bay Area with CZ Biohub's innovative team to catalyze impact and form partnerships that benefit people globally. We strive to make our tools and technologies available to scientists worldwide, free of charge.

The organization is maintained by Loic Royer and Jordao Bragantini. Please contact Loic, Jordao, or other members of the team with your questions. If you have a bug report or question about one of our projects, please open an issue on that project.

Interested in working with us? Search our job openings here.

Latest news from the Royer Lab can be found on Loic's twitter account

Some repositories in this org predate Loic's joining CZB SF as group leader. Other organisations have been created and in some cases still maintained by Loic or other members of Royer lab:

• The ClearVolume organisation holds repositories created by Loic Royer in the context of the ClearVolume paper when he was a postdoc in the Gene Myers lab at MPI CBG.
• The AutoPilot organisation holds repositories created by Loic Royer in the context of the Autopilot paper when he was collaborating as a postdoc with Philipp Keller at HHMI Janelia.