3D Docs #411

ilan-gold · 2021-04-07T17:24:03Z

For #398 this adds documentation on how our coordiante systems and affine transformations work. This is really just a first pass and I'm probably not conveying everything clearly but I am pretty confident about the transformation at the bottom, which is probably the main thing people will end up taking away from this. Please feel free to completely take this apart.

Change List

Add a 3d rendering doc

Checklist

Update JSdoc types if there is any API change.
Make sure Avivator works as expected with your change.

manzt

Thanks, Ilan. This is really detailed. Thanks for taking the time to write this up!

However, I want to push back slightly on dumping this into our documentation. I think it is fine to keep in the Viv repo as a reference for developers, but it is a bit too involved in my opinion to be useful to Viv users. It's clear you have thought about this a lot and understand it deeply, and that is reflected in the text. I'm just not sure what level of detail is ultimately practical/useful to most.

What I'd like to see in our docs that helps the user with the question "I did this in napari, why isn't it working here?". Maybe a short paragraph that touches on the following points:

Similar to napari, Viv implements ray-casting on in-memory volumetric data, and exposes an API for applying arbitrary affine transformations to the volume when rendering.
Viv expects a traditional graphics affine matrix that is x,y,z along the diagonal (maybe draw the matrix).
An affine matrix used in napari is typically ordered z,y,x to match broadcasting conventions with numpy.
Minimal example, transforming numpy/napari transformation matrix to viv matrix. Little text.

manzt · 2021-04-08T14:05:06Z

docs/3D_RENDERING.md

@@ -0,0 +1,92 @@
+## 3D Rendering
+
+Viv has the capbility to do volume ray-casting on in-memory volumetric data. It also exposes an API for applying arbitrary affine transformations to the volume when rendering. Napari is another popular tool for doing both of these, but there are some key differences. Viv follows the [convention of 3D graphics](https://northstar-www.dartmouth.edu/doc/idl/html_6.2/Coordinates_of_3-D_Graphics.html) to have the x-axis running left-right across the screen, the y-axis running up and down, and the z-axis running into and out of the screen (all following the right-hand-rule for standard orientation). Napari, by contrast, has a different way of orienting volumes for two main reasons: first, it is labeless with no knowledge of what the user cosniders to be the `z` axis of the data and second, it relies on `numpy` for in memory representation of the data.


considers

it would be useful to use semantic line breaks so this is easier to edit.

manzt · 2021-04-08T14:07:25Z

docs/3D_RENDERING.md

+
+### Coordinate System
+
+As mentioned, Viv follows the [convention of 3D graphics](https://northstar-www.dartmouth.edu/doc/idl/html_6.2/Coordinates_of_3-D_Graphics.html) with respect to the x, y, and z axes. These axes correspond exactly to the dimensions of the OME model which are labelled on the data loaders that Viv expoerts.


Could you run a spell checker? I'm likely missing things as well.

manzt · 2021-04-08T14:10:34Z

docs/3D_RENDERING.md

+#        0., 0., 0., 2., 0., 0., 0., 0., 2., 0., 0., 0., 0., 2.])
+```
+
+However, if Napari wishes to show the volume with the correct orientation while respecting this in-memory represnetation, it is forced to buck the 3D graphics convention. As in the 2D case, Napari actually continues to treat the y-axis going down as positively oriented (as mentioned, the 3D graphics convention for the y-axis runs contrary to the convention in 2D raster graphics, where the y-axis is positively oriented in the downward direction). That is, in Napari, the `[0,0,0]` coordinate of your volumetric data array corresponds to the `(0,0,0)` origin in the 3D visualization space. However, because Viv maintains the graphics convention, the `[0,0,0]` data point (and all other data points) must be anti-diagonally transposed in order to maintain the correct orientation of the volume. That is, if you have a `[k,n,m]` shaped data cube in `numpy`, and you wanted to visualize it in Viv via `Zarr`, for example, the `[0,n,m]` data point is actually at the origin `(0,0,0)` and the `[0,0,0]` data point is at `(0,n,m)`. If Viv did not do this anti-diagonal transposition, the volume would look upside down because the `[0,0,0]` data point would be at `(0,0,0)`, which is the "bottom" of the volume.


Pretty lost here. I still don't think I understand this anti-diagonal reasoning, but you've thought much more about this than myself.

ilan-gold · 2021-04-08T14:31:25Z

@manzt That makes perfect sense. I will do that, taking the short snippet at the end and some text, and putting that in our docs, and putting the rest in our repo, like IMAGE_RENDERING.md.

manzt

Great, thanks!

docs/3D_RENDERING_BRIEF.md

Co-authored-by: Trevor Manz <trevor.j.manz@gmail.com>

* 3D Viv Core (#409) * Initial commit * CHANGELOG. * Address comments. * Prettier. * 3D Docs (#411) * First pass at docs. * Clean up. * Add to docs. * Spelling & brief docs. * Change docs target * Update docs/3D_RENDERING_BRIEF.md Co-authored-by: Trevor Manz <trevor.j.manz@gmail.com> Co-authored-by: Trevor Manz <trevor.j.manz@gmail.com> * Fix 3D types/docs * Remove header. * Basic 3D Avivator (#412) * Initial commit * CHANGELOG. * Address comments. * Prettier. * [WIP] Avivator first pass. * Fix webpack config. * Fix restricted syntax. * 3D Non-Orthogonal Clipping Planes (#418) * Initial commit * CHANGELOG. * Address comments. * Prettier. * [WIP] Avivator first pass. * Add non-orthogonal clipping planes. * Use `flat` * Fix typo * Fix numPlanes padding * Fix plane equation. * Use sign function instead of ceil * Remove cast. * Use Plane API * Remove `numPlanes` argument. * (For 3D) Avivator Refactor (#420) * Initial commit * CHANGELOG. * Address comments. * Prettier. * [WIP] Avivator first pass. * [WIP] State management, some updates to Avivator. * [WIP] Adding more use of the global state. * [WIP] More state replacement. * Change color palette selector. * Use imgae settings. * Use viewer store. * Clean up unnecessary Avivator props. * Create self-contained buttons. * Refactor main `useEffect` to use hooks. * Fix snackbars. * Refactor Viewer. * Change names of buttons. * Refactor Snackbars * Remove unused vars. * Refactor menu. * Reset with new image. * Fix linting error. * Refactor props -> global state for dropzone components. * Remove more unnecessary props. * Colorpalette use global state. * Refactor selections. * Fix async selection bug. * Fix viewState/initialViewState issue. * Fix toggle. Remove dead code. * Small ids fix. * Refactor channel stats. * Use > instead of epsilon. * Revert loader change. * Fix import paths. * Tweaks. * Prettier. * Fix state bug and lens display. * Handle undefined loader in state. Small entries suggestion. * Reform state management. * Small code compleixty change. * Fix artifacts for additive. * Small Features/Fixes for Viv Src (#422) * Fixes for viv src * Update src/layers/ImageLayer.js Co-authored-by: Trevor Manz <trevor.j.manz@gmail.com> * Prettier, Co-authored-by: Trevor Manz <trevor.j.manz@gmail.com> * New Avivator (#421) * Initial commit * CHANGELOG. * Address comments. * Prettier. * [WIP] Avivator first pass. * [WIP] State management, some updates to Avivator. * [WIP] Adding more use of the global state. * [WIP] More state replacement. * Change color palette selector. * Use imgae settings. * Use viewer store. * Clean up unnecessary Avivator props. * Create self-contained buttons. * Refactor main `useEffect` to use hooks. * Fix snackbars. * Refactor Viewer. * Change names of buttons. * Refactor Snackbars * Remove unused vars. * Refactor menu. * Reset with new image. * Fix linting error. * Refactor props -> global state for dropzone components. * Remove more unnecessary props. * Colorpalette use global state. * Refactor selections. * Fix async selection bug. * Fix viewState/initialViewState issue. * Fix toggle. Remove dead code. * Small ids fix. * Refactor channel stats. * Use > instead of epsilon. * Revert loader change. * Fix import paths. * Tweaks. * Prettier. * Styling. * Slicers using polar coordinates. * Merge branch 'ilan-gold/avivator_refactor' into ilan-gold/new_avivator * Use epsilon value. * Use better range of theta. * onViewportLoad callbacks. * Remove plane flip * Move onViewportLoad to image settings. * Set up isLoading callbacks. * Fix AddChannel call backs. * Allow options to be disabled as well. * Simplify global selections * AbortController for volume layer * Only do keydown change selection * Add delay to onViewportload prop for volumes * Add slicer presets. Move slicer location. * Fix Camera Axis. * Debounce view state change. * Fix coordinate system for volume. * Change scaling of normal planes. * Linting. * Fix old merge issues * Change coordinates of slicing * Add isLoading for channel. * Add isChannelLoading removal. * Chane when channel is set to loading. * Fix jsdocs. Small fix for volume button. * Don't always use physical size scaling. * Fix loader physical size inference. * Make typescript happy. * Small fix. * Fix properties state bug. * Fix rotation recentering. * Use copy of clipping planes. * Fixes for viv src * Address comments. * Slicing Reversion + New Demo Datasets (#430) * Revert to orthogonal slicing. * Add idr datasets * Fix slicer styling. * Small doc fixes. * Add interpolation to 3D * Add check for interpolation. * prettier. * Final Changes for Avivator (#434) * Use tabs for controller. * WRong disabled cpondition for volume button * Change non-3d rendering mode display. * Small style tweaks. * Fix checkbox. Small styling issues. * Rename * Add disabled styling. * Add spacing. * Fix slicer styling. * Set bounding cube intiially. * Add warning about WebGL2 * Remind to refresh page. * Fix for 3D Blending. (#440) * Minimal Fix for 3D Blending. * Remove culling on layer * Add comment. * Prettier. * Add progress indicator option + onUpdate callback. (#446) Co-authored-by: Trevor Manz <trevor.j.manz@gmail.com>

ilan-gold added 3 commits April 7, 2021 12:42

First pass at docs.

0b30ecc

Clean up.

131a55e

Add to docs.

5f0d27f

ilan-gold changed the base branch from master to feature/3d April 7, 2021 17:24

ilan-gold requested a review from manzt April 7, 2021 17:24

manzt reviewed Apr 8, 2021

View reviewed changes

ilan-gold added 2 commits April 8, 2021 11:48

Spelling & brief docs.

147a6e3

Change docs target

b5959d1

ilan-gold requested a review from manzt April 8, 2021 15:52

manzt approved these changes Apr 9, 2021

View reviewed changes

docs/3D_RENDERING_BRIEF.md Outdated Show resolved Hide resolved

Update docs/3D_RENDERING_BRIEF.md

44cd9a7

Co-authored-by: Trevor Manz <trevor.j.manz@gmail.com>

ilan-gold merged commit 13c9ee4 into feature/3d Apr 9, 2021

ilan-gold deleted the ilan-gold/3d_docs branch April 9, 2021 14:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

3D Docs #411

3D Docs #411

ilan-gold commented Apr 7, 2021 •

edited

Loading

manzt left a comment •

edited

Loading

manzt Apr 8, 2021

manzt Apr 8, 2021

manzt Apr 8, 2021

manzt Apr 8, 2021

manzt Apr 8, 2021

manzt Apr 8, 2021

ilan-gold commented Apr 8, 2021

manzt left a comment

		@@ -0,0 +1,92 @@
		## 3D Rendering

		Viv has the capbility to do volume ray-casting on in-memory volumetric data. It also exposes an API for applying arbitrary affine transformations to the volume when rendering. Napari is another popular tool for doing both of these, but there are some key differences. Viv follows the [convention of 3D graphics](https://northstar-www.dartmouth.edu/doc/idl/html_6.2/Coordinates_of_3-D_Graphics.html) to have the x-axis running left-right across the screen, the y-axis running up and down, and the z-axis running into and out of the screen (all following the right-hand-rule for standard orientation). Napari, by contrast, has a different way of orienting volumes for two main reasons: first, it is labeless with no knowledge of what the user cosniders to be the `z` axis of the data and second, it relies on `numpy` for in memory representation of the data.


		### Coordinate System

		As mentioned, Viv follows the [convention of 3D graphics](https://northstar-www.dartmouth.edu/doc/idl/html_6.2/Coordinates_of_3-D_Graphics.html) with respect to the x, y, and z axes. These axes correspond exactly to the dimensions of the OME model which are labelled on the data loaders that Viv expoerts.

3D Docs #411

3D Docs #411

Conversation

ilan-gold commented Apr 7, 2021 • edited Loading

Change List

Checklist

manzt left a comment • edited Loading

Choose a reason for hiding this comment

manzt Apr 8, 2021

Choose a reason for hiding this comment

manzt Apr 8, 2021

Choose a reason for hiding this comment

manzt Apr 8, 2021

Choose a reason for hiding this comment

manzt Apr 8, 2021

Choose a reason for hiding this comment

manzt Apr 8, 2021

Choose a reason for hiding this comment

manzt Apr 8, 2021

Choose a reason for hiding this comment

ilan-gold commented Apr 8, 2021

manzt left a comment

Choose a reason for hiding this comment

ilan-gold commented Apr 7, 2021 •

edited

Loading

manzt left a comment •

edited

Loading