recorder reconstruct CLI

[talonchandler]

@ziw-liu @mattersoflight and I have been brainstorming about a key set of features to prepare ahead of AQLM.

We've roughly settled on providing a recorder reconstruct CLI that lets the user reproduce what they've done in the GUI. Followup tweaking of reconstruction parameters can be performed by modifying a .yaml file that is saved after each recOrder GUI acquisition.

We're imagining the following workflow:

1. Calibration proceeds like usual in the "Calibration" tab and results are saved in calibration_metadata.txt

2. The user acquires and reconstructs data in the "Acquire" tab, and iterates on the microscope until they're happy with their data.

3. After each acquisition is complete, recOrder saves all of its parameters to a gui_state.yml file and prints a one-line CLI call that can be used to reproduce the reconstruction that they've been shown in the napari GUI. The CLI call will look like:
   recorder reconstruct --input=/path/to/data.zarr --output=/path/to/output.zarr --params=3dphase,3dbiref --config=gui_state.yml

4. The user can modify the gui_state.yml file to change the reconstruction parameters (regularization parameters, etc).

#279 already saves the gui_state.yml file with enough information to recreate the reconstruction, so we're planning to modify the example scripts into a CLI that reads the gui_state.yml file.

We're very open to feedback on this plan, which is our main issue to solve for 0.4.0. A few specific questions:

• Is the proposed CLI call clear? --params=3dphase,3dbiref?
• We're thinking of giving the user the option to append to an existing zarr (enabled by iohub!), possibly by setting the output equal to the input. Good idea?
• We're planning to have the CLI fail early. We'll validate file i/o, check the input/output shapes for the requested params, and validate the config file before proceeding to the actual computation.
• recOrder is currently set up around single-position datasets. @ziw-liu pointed out that these CLI calls could easily be adapted to multi-position files (acquired via MM then converted to zarr) by using the nested structure of iohub zarrs.

[mattersoflight]

I'd add that this CLI will use the new unified polarization algebra and deconvolution algorithms being developed in waveOrder. This CLI will seed the design of the offline analysis GUI we plan to have in release 1.0.0 of recOrder.

Some thoughts on CLI

• I'd rename gui_state.yml to microscope_params.yml, microscope_state.yml or recorder_params.yml
• Instead of --params=3dphase,3dbiref, --channels= 3dphase,3dbiref is more intuitive.
• For example, following CLI is more intuitive to me.
  recorder reconstruct --channels=3dphase,3dbiref --input=/path/to/data.zarr --output=/path/to/output.zarr --config=microscope_params.yml

• We're thinking of giving the user the option to append to an existing zarr (enabled by iohub!), possibly by setting the output equal to the input. Good idea?

Yes! It does make sense to write the reconstructed data as channels that can be read with fluorescence or raw transmission data. A workflow for the user will be

• convert the TIFF stack (or another data format) into ome-zarr.
• add reconstructions to the ome-zarr.

• We're planning to have the CLI fail early. We'll validate file i/o, check the input/output shapes for the requested params, and validate the config file before proceeding to the actual computation.
• recOrder is currently set up around single-position datasets. @ziw-liu pointed out that these CLI calls could easily be adapted to multi-position files (acquired via MM then converted to zarr) by using the nested structure of iohub zarrs.

👍🏼 to both of the above.

[talonchandler]

#347 addresses this, and we're now finishing up details and testing a near variant of this CLI design. Closing.