Implement O3 autofocus on light-sheet arm #48

edyoshikun · 2023-06-21T23:19:34Z

This PR introduces O3 refocus mechanism to compensate for drift in the O3 position which leads to defocus of images from the LS RR path.

O3 autofocus is achieved by moving the O3 KIM101 stage to axially displace the objective and then using the waveorder.focus.focus_from_transverse_band algorithm to find the in-focus slice.

Several criteria are implemented to ensure that O3 does not move too close to O2:

Focus is evaluated at 3 different galvo positions as the LS RR path images only a small section of the sample, which may be blank, at a given galvo position
A minimum FWHM threshold is used when evaluating the focus metric, as described in Add optional parameters plot_path and threshold_FWHM to focus_from_transverse_band mehta-lab/waveorder#132, to reject blank stacks
A conservative travel limit on O3 is implemented, allowing the objective to move only ~10 um towards O2 or ~20 um away from O2.

The travel limits we have currently implemented are safe, but when coupled with hysteresis of the O3 stage, limit the total duration of time lapse experiments. In a future PR we can extend the duration of time lapse experiments by:

Better calibrating the O3 stage to remove hysteresis
Developing even more fool-proof focus finding algorithms which will always keep O3 is focus, thus a safe distance from O2, or will not move O3 if the sample dies / disappears. The current implementation strives for that, but needs to be extensively tested before we can remove the additional travel limit safeguards.

Resolves #19

edyoshikun · 2023-06-21T23:26:45Z

pyproject.toml

@@ -31,6 +31,7 @@ dependencies = [
    "nidaqmx",
    "iohub==0.1.0.dev3",
    "matplotlib",
+    "pythonnet"


actually this should be part of copylot.

into drift_correction

ieivanov · 2023-07-21T21:09:09Z

@edyoshikun it will be great to get your review too, I can't formally request it because you opened this PR

mattersoflight

I am approving based on the stability tests @ieivanov has reported. @edyoshikun can't formally be the reviewer, but Ed should run the code himself to check for reproducibility.

talonchandler

LGTM! I only have a pair of minor non-blocking comments.

talonchandler · 2023-07-25T00:48:26Z

examples/test_kim101_stage.py

Did I hear correctly that this file wasn't/isn't working? Maybe add a comment, and consider renaming to test_kim101_copylot.py to match test_kim101_pylablib.py.

talonchandler · 2023-07-25T01:02:18Z

mantis/acquisition/acq_engine.py

+        # Refocus O3
+        # Some focus_indices may be None, e.g. if there is no sample
+        valid_focus_indices = [idx for idx in focus_indices if idx is not None]
+        if valid_focus_indices:


Cool, I didn't know that empty sequences return false. If this is common knowledge, great! If not, maybe comment?

I think it's relatively common knowledge: https://stackoverflow.com/questions/53513/how-do-i-check-if-a-list-is-empty

edyoshikun · 2023-07-21T21:17:29Z

examples/test_kim101_stage.py

+# %%
+from copylot.hardware.stages.thorlabs.KIM001 import KCube_PiezoInertia
+import time
+from copylot import logger


unused imports logger and time

edyoshikun · 2023-07-21T21:57:38Z

mantis/acquisition/acq_engine.py

+            idx = focus_from_transverse_band(
+                stack,
+                NA_det=NA_DETECTION,
+                lambda_ill=wavelength,


should we make this also a global for now?

edyoshikun · 2023-07-21T21:58:16Z

mantis/acquisition/acq_engine.py

+                NA_det=NA_DETECTION,
+                lambda_ill=wavelength,
+                pixel_size=LS_PIXEL_SIZE,
+                threshold_FWHM=threshold_FWHM,


this one as global or at the top of the file?

edyoshikun · 2023-07-21T22:05:54Z

mantis/acquisition/acq_engine.py

+        if np.any(target_z_position > max_z_position) or np.any(
+            target_z_position < min_z_position
+        ):
+            logger.error('O3 relative travel limits will be exceeded. Aborting O3 refocus.')


should this raise Exception or with the logger is fine?

Here we don't want to raise an Exception as that will terminate the acquisition. Our current model is to continue imaging even if O3 autofocus fails. We can consider changing that once we have more trust in the autofocus algorithm such that it quits the acquisition once O3 has reached it's travel limit and cannot be refocused anymore.

edyoshikun · 2023-07-21T23:58:12Z

mantis/acquisition/acq_engine.py

+        # Define O3 z range
+        # 1 step is approx 20 nm, 15 steps are 300 nm which is sub-Nyquist sampling
+        # The stack starts away from O2 and moves closer
+        z_start = -105


perhaps we should rename these z_variables to o3_z_start. I assume this is a range you found to be good for scanning, but should these variables be globals or parsed from another file?

Renamed as suggested.

Let's see if these values need to change at all. If they are effectively constants, then we can leave them as local variables in the refocus_ls_path function - they are not used by other functions and I think they'll be easier to find here.

edyoshikun · 2023-07-26T20:40:13Z

mantis/acquisition/microscope_operations.py

+    stage : KinesisPiezoMotor
+
+    """
+    logger.debug(f'Setting up Kinesis Piezo Motor stage with serial number {serial_number}')


I assume that the KIM101 has it's own guardrails to know what th maximum_voltage,velocity, and acceleration are. In our case, should we set a separate restriction of a range of velocity and acceleration values that we have calibrated the stage? I think that we should restrict some of these parameters for our usecase.

We don't really expose any way for the users to change these parameters, so I think we can leave this for now

* adding pythonnet and drift correction * removing the pythonnet dependency that should live in copylot * cleanup the demo code * move drift_correction.py to examples * add copylot to dependencies * add O3 stage setup * move serial numbers to acq_engine * rename drift_correction script * add defocus stack acquisition * style * update examples * refactor z stage move * add galvo scanning to acquire_ls_defocus_stack * Update acquire_defocus_stack.py * update examples * switch to pylablib stage and relative moves * add some logging * add KIM101 compensation * fix galvo reset position bug * add find_focus.py example * style * ignore tests in examples dir * move o3 refocus to acq_engine * add checks and logging * debug acq engine o3 refocus * update example * Create separate logs directory * move conda env logger to logs * rename to mantis_acquisition_log * save acquired stacks * implement timed o3 refocus * update kim101 calibration * add relative O3 travel limits * logger fixes * style * add waveorder to deps and format pyproject * update kim101 compensation factor * add threshold and plotting to focus finding * update waveorder dependency * add microscope_operations documentation * update data structure specs * make acquire_ls_defocus_stack MantisAcquisition static method * update examples * rename_kim101 example * rename z_range vars to avoid confusion --------- Co-authored-by: Uditha Velidandla <ivan.ivanov@czbiohub.org>

* adding first prototype of manual registration with napari * adding layer control to hide and show the aligned dataset. * replacing fluor/phase to respective dataset names and saving metadta * fix typo on * fixing channel mixup and adding 3D registration. * z-sampling typo. channels were flipped * fixed focus channel parameters and making this a function to save only the parameters * fixing layer typo * reverting accidental commit with some test * tet for registration.py * adding registration function * adding shape metadata * adding the CLI for applying registration with multiprocessing * adding registration parameter output a more meaningful name * replacing affine transform to similarity transform. this avoids shearing. * adding voxel_size * handle coordinate transformation * formatting isort and black * removing own registration algorithm as scikit image one works * fixes to use only click.echo rather than print * handling float inputs * renaming file to register.py to register and deleting the redundant registration.py * adding refactored register and utils function * renaming chan_1 and chan_2 to phase and fluor * fixing typos to add registration to cli * adding parameters for register.cli that live in utils.py * Fix_deskew_utils (#67) * replacing functinos that live in utils.py * adding utils.py to slurmkit file * adding validation and sorting to parsing.py * update parsing.py for deskew * adding the metadata per position * adding the yaml file writing and parsing * fixing deskew function new utils.py * slurm deskew fixed with new util.py * adding slumkit for registration * Implement O3 autofocus on light-sheet arm (#48) * adding pythonnet and drift correction * removing the pythonnet dependency that should live in copylot * cleanup the demo code * move drift_correction.py to examples * add copylot to dependencies * add O3 stage setup * move serial numbers to acq_engine * rename drift_correction script * add defocus stack acquisition * style * update examples * refactor z stage move * add galvo scanning to acquire_ls_defocus_stack * Update acquire_defocus_stack.py * update examples * switch to pylablib stage and relative moves * add some logging * add KIM101 compensation * fix galvo reset position bug * add find_focus.py example * style * ignore tests in examples dir * move o3 refocus to acq_engine * add checks and logging * debug acq engine o3 refocus * update example * Create separate logs directory * move conda env logger to logs * rename to mantis_acquisition_log * save acquired stacks * implement timed o3 refocus * update kim101 calibration * add relative O3 travel limits * logger fixes * style * add waveorder to deps and format pyproject * update kim101 compensation factor * add threshold and plotting to focus finding * update waveorder dependency * add microscope_operations documentation * update data structure specs * make acquire_ls_defocus_stack MantisAcquisition static method * update examples * rename_kim101 example * rename z_range vars to avoid confusion --------- Co-authored-by: Uditha Velidandla <ivan.ivanov@czbiohub.org> * Update recommended sampling in the LS and LF remote refocus paths (#69) * Update recommended sampling in the LS and LF RR paths Co-Authored-By: Talon Chandler <talonchandler@gmail.com> * Drop mirror scan step by half Co-authored-by: Talon Chandler <talonchandler@gmail.com> --------- Co-authored-by: Talon Chandler <talonchandler@gmail.com> * Improve autofocus reengagement (#74) wait for autofocus to engage * All CLI calls read from zarr (#71) * read from zarr * Use better colors via `iohub` * isort * `estimate_deskew` read from zarr * first-pass standardize register * remove defunct test * fix readme. * standardize estimate_registration * typo * show `mantis -h` in order * standardize register cli * fix parameter * fix estimate-bleaching bug * fix deskew tests * flake8, isort * add registration tests * Update mantis/analysis/AnalysisSettings.py * reducing parameters in config file * re-adding finding best focus slice and rotate image by 90deg * removing test main calls * fixing apply_affine to perform a rotation first before affine. * renaming cli calls and updating README * renaming files for readability and to match CLI * style fix * adding tests and renaming variables for clarity * missed changin the conftest.py * fixing typo on test for test appy cli * modifying code to pass tests * deleting unsed imports flake8 * fix incorrect click.echo * adding pre-affine 90 deg rotationsa bout z as option for cli and croping ROI for focus finding * swapping config to use the indentity matrix as the affine matrix * return the affine transfrom mapping phase to fluorescence * fixing pre_affine_90degree_rotations_about_z in config * renaming and removing redundant printf() * update slurmkit deskew * adding function to read the voxel size from metadata and updating registration slurmkit * voxel_size retreival and modifing apply_affine to be applied to the labelfree channel. * fix typo * updating slurmkit apply_affine * cleanup slurmkit * pass voxel sizes through apply_affine * scaling passes through apply_affine tested * isort * update readme --------- Co-authored-by: Uditha Velidandla <ivan.ivanov@czbiohub.org> Co-authored-by: Talon Chandler <talonchandler@gmail.com>

adding pythonnet and drift correction

f98ebd0

edyoshikun commented Jun 21, 2023

View reviewed changes

edyoshikun and others added 28 commits June 21, 2023 16:31

removing the pythonnet dependency that should live in copylot

541b816

cleanup the demo code

e75eb99

move drift_correction.py to examples

6e6b2f1

add copylot to dependencies

23d9344

add O3 stage setup

2f83900

move serial numbers to acq_engine

9d6a059

rename drift_correction script

dd226de

add defocus stack acquisition

7d3b0d0

style

4a805c4

update examples

cbf624d

refactor z stage move

391bf17

Merge branch 'drift_correction' of https://github.com/czbiohub-sf/mantis

a50cc24

into drift_correction

add galvo scanning to acquire_ls_defocus_stack

638f384

Update acquire_defocus_stack.py

5721485

update examples

4465b49

switch to pylablib stage and relative moves

82ad026

add some logging

94f47b7

add KIM101 compensation

f143017

fix galvo reset position bug

a5ef657

add find_focus.py example

276abf6

style

46e04de

ignore tests in examples dir

6372a18

move o3 refocus to acq_engine

702e8f3

add checks and logging

211b49d

debug acq engine o3 refocus

b3b34e7

update example

838e77b

Create separate logs directory

9177c87

Merge branch 'main' into drift_correction

56a4fc4

ieivanov added 13 commits July 12, 2023 17:17

implement timed o3 refocus

106609e

update kim101 calibration

b263cff

add relative O3 travel limits

32d1857

logger fixes

e5a3e3a

style

50ae9a1

add waveorder to deps and format pyproject

7c1e4e3

update kim101 compensation factor

9950c50

add threshold and plotting to focus finding

04a35c6

update waveorder dependency

8b580df

add microscope_operations documentation

f4ab651

update data structure specs

badfdf6

make acquire_ls_defocus_stack MantisAcquisition static method

b5e9766

update examples

8fbfa31

ieivanov requested review from talonchandler and mattersoflight July 21, 2023 21:06

ieivanov marked this pull request as ready for review July 21, 2023 21:08

ieivanov changed the title ~~Drift correction for lightsheet~~ Implement O3 autofocus on light-sheet arm Jul 21, 2023

ieivanov mentioned this pull request Jul 21, 2023

Develop image-based autofocus for LF arm #63

Open

mattersoflight approved these changes Jul 24, 2023

View reviewed changes

talonchandler approved these changes Jul 25, 2023

View reviewed changes

rename_kim101 example

1f07549

edyoshikun commented Jul 26, 2023

View reviewed changes

talonchandler mentioned this pull request Jul 27, 2023

Add more testing for missing variables for Channel Settings. #40

Closed

rename z_range vars to avoid confusion

15e9ee0

ieivanov merged commit 1810fa3 into main Jul 27, 2023

ieivanov deleted the drift_correction branch August 19, 2023 00:38

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement O3 autofocus on light-sheet arm #48

Implement O3 autofocus on light-sheet arm #48

edyoshikun commented Jun 21, 2023 •

edited by ieivanov

Loading

edyoshikun Jun 21, 2023

ieivanov commented Jul 21, 2023

mattersoflight left a comment •

edited

Loading

talonchandler left a comment

talonchandler Jul 25, 2023

talonchandler Jul 25, 2023

ieivanov Jul 25, 2023

edyoshikun Jul 21, 2023

edyoshikun Jul 21, 2023

edyoshikun Jul 21, 2023

edyoshikun Jul 21, 2023

ieivanov Jul 27, 2023

edyoshikun Jul 21, 2023

ieivanov Jul 27, 2023

edyoshikun Jul 26, 2023

ieivanov Jul 27, 2023

Implement O3 autofocus on light-sheet arm #48

Implement O3 autofocus on light-sheet arm #48

Conversation

edyoshikun commented Jun 21, 2023 • edited by ieivanov Loading

Choose a reason for hiding this comment

ieivanov commented Jul 21, 2023

mattersoflight left a comment • edited Loading

Choose a reason for hiding this comment

talonchandler left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

edyoshikun commented Jun 21, 2023 •

edited by ieivanov

Loading

mattersoflight left a comment •

edited

Loading