Improve `log_pinhole` and support non-RDF pinholes #2614

emilk · 2023-07-05T21:39:05Z

Closes #2244, closes #2589

What

log_pinhole is now much easier to use. You only need to set focal length and optional principal point instead of setting the full 3x3 matrix.

There is also a new argument to it camera_xyz for setting the coordinate system. The default is RDF (the old default). This affects the visible camera frustum, how rays are projected when hovering a 2D image, and how depth clouds are projected.

There is also a bunch of bug fixes along the way, including an accurate camera frustum which works for non-centered principal points.

This simplifies some examples, especially examples/python/open_photogrammetry_format/main.py which uses RUB.

Tests

There is a new test: tests/python/view_coordinates/main.py creates a bunch of pinholes with RGB and Depth.

I've also run

Checklist

I have read and agree to Contributor Guide and the Code of Conduct
I've included a screenshot or gif (if applicable)
I have tested demo.rerun.io (if applicable)

wip

…nates

…ndo subtle change in default eye orientation

…iew-coordinates

jleibs · 2023-07-06T12:52:11Z

crates/re_components/src/coordinates.rs

+    /// Default right-handed view coordinates of `re_renderer`: X=Right, Y=Up, Z=Back.
+    pub const RUB: Self = Self([ViewDir::Right, ViewDir::Up, ViewDir::Back]);


Is there a hard constraint forcing this, or do we have the option to (in the future) make the default view coordinates of re_renderer RDF instead? This seems like it would cut down on some of the book-keeping around whether a camera is a "real camera" vs a "re-renderer camera." Having one standard set of conventions for cameras would reduce ambiguity, and given that we position ourselves as a robotics-and-computer-vision product, I think RDF is the right choice here.

I agree it would be nice to use RDF as the default everywhere. We could probably switch re_renderer with some work, though it should be noted that RDF is quite unusual for rendering. @Wumpf can probably add some more nuance here.

jleibs · 2023-07-06T13:19:28Z

crates/re_space_view_spatial/src/space_camera_3d.rs

+        // View-coordinates are relevant here because without them we have no notion of what the image plane is.
+        // (it's not a given that e.g. XY is the camera image plane!)
+        // First transform to the "standard RUB" 3D camera and then from there to the image plane coordinate system.


Suggested change

// View-coordinates are relevant here because without them we have no notion of what the image plane is.

// (it's not a given that e.g. XY is the camera image plane!)

// First transform to the "standard RUB" 3D camera and then from there to the image plane coordinate system.

// The pinhole view-coordinates are important here because they define how the image plane is aligned

// with the camera coordinate system. It is not a given that a user wants the image-plane aligned with the

// XY-plane in camera space.

//

// Because the [`Pinhole`] component currently assumes an input in the default `image_view_coordinates`

// we need to pre-transform the data from the user-defined `pinhole_view_coordiantes` to the required

// `image_view_coordinates`. We do this by converting the points into and out of RUB.

//

// TODO(...): When Pinhole is an archetype instead of a component, `pinhole.project` should do this

// internally.

gripe: still don't love RUB being the "standard" here.

a.from_rub() * b.to_rub() would be equivalent to a.from_rdf() * b.to_rdf(). I'd be happy to change to the latter in most places. We could even though a.from_other(b) to not even have an inbetween step

Done in 62fad9c

jleibs · 2023-07-06T13:25:18Z

crates/re_space_view_spatial/src/scene/parts/cameras.rs

@@ -207,10 +177,10 @@ impl ScenePart<SpatialSpaceView> for CamerasPart {
            let query = re_arrow_store::LatestAtQuery::new(query.timeline, query.latest_at);

            if let Some(pinhole) = store.query_latest_component::<Pinhole>(ent_path, &query) {
-                let view_coordinates = determine_view_coordinates(
+                let pinhole_view_coordinates = pinhole_camera_view_coordinates(


Can't wait for Archetype-queries. Would be much less error-prone to have this be an associated component of the Pinhole archetype. Then we could account for it inside of Pinhole.project instead of needing to pre-condition the data before calling the project function.

Yeah, it would be much nicer. I also look forward splitting ViewCoordinates into several different components

docs/content/concepts/spaces-and-transforms.md

jleibs · 2023-07-06T13:58:36Z

examples/python/blueprint/main.py

Why is this an empty file...?

The file isn't empty - the diff is. Or rather: the diff is that I did fd main.py | xargs chmod +x because I don't have the patience of writing python in the terminal when running scripts :)

jleibs · 2023-07-06T13:59:31Z

examples/python/arkit_scenes/main.py

@@ -254,8 +254,7 @@ def log_camera(
    # pathlib makes it easy to get the parent, but log methods requires a string
    camera_path = str(PosixPath(entity_id).parent)
    rr.log_transform3d(camera_path, camera_from_world, from_parent=True)
-    rr.log_view_coordinates(camera_path, xyz="RDF")  # X=Right, Y=Down, Z=Forward
-    rr.log_pinhole(f"{entity_id}", child_from_parent=intrinsic, width=w, height=h)
+    rr.log_pinhole(entity_id, width=w, height=h, child_from_parent=intrinsic)


did you want to add a "camera_xyz" here?

I'm not sure.

We can either add camera_xyz=RDF to all examples to be explicit. This could either be taken as an encouragement to users to use RDF (good), or as an invitation to change it to something else (perhaps less good).

If we instead omit it when not needed it could lead to questions of what the default is and how to change it.

examples/python/rgbd/main.py

jleibs · 2023-07-06T14:38:22Z

rerun_py/rerun_sdk/rerun/log/camera.py

+        NOTE: setting this to something else than "RDF" (the default) will change the orientation of the camera frustum,
+        and make the pinhole matrix not match up with the coordinate system of the pinhole entity.
+        The pinhole matrix (the `child_from_parent` argument) always project along the Z axis of the camera space,
+        but will be re-oritented to project along another axis if the `camera_xyz` argument is set.


This definition of "re-oriented" is vague as it doesn't do a great job of describing what happens to the direction of pixel-x and pixel-y directions. I believe in essence this is more of "remapping" the axes than re-orienting (which will be important once we allow right-to-left conversions).

I believe the actual definition of this is:

Whichever axis is labeled F will be the positive projection direction (B means negative projection values are "into" the image).

Whichever axis is labeled R is positive pixel-x (increasing column index). L means we flip the image horizontally.

Whichever axis is labeled D is the positive pixel-y (increasing row index). U means we flip the image vertically.

I guess L / U doesn't actually mean we flip the image exactly... but we flip the indexing of the image. So increasing axis = drecreasing pixel-x from "right" to "left" across the image (which is technically the same as increasing pixel coordinates across a flipped copy of the image).

I've expanded the docstring

rerun_py/rerun_sdk/rerun/log/camera.py

Co-authored-by: Jeremy Leibs <jeremy@rerun.io>

emilk · 2023-07-07T14:32:42Z

Ok, I think this is good enough now.

I'm not 100% married to this idea of implicit transforms, but I think the overall code is a lot clearer now, and changing to another approach will yield a much smaller diff.

In particular we have:

A great test
Improved log_pinhole ui
An easy-to-use interface for when we don't want to use RDF (e.g. the open_photogrammetry_format example)
Much more explicit Rust code
Encouraging users to use the camera_xyz argument of log_pinhole instead of using the free-standing log_view_coordinates function.
Simpler and bug-free frustum code

@kpreid

…nts - 2023-07-27 (#2842) ## [0.8.0](v0.7.0...v0.8.0) - Infrastructure investments and more transform improvements - 2023-07-27 [Rerun](https://www.rerun.io/) is an easy-to-use visualization toolbox for computer vision and robotics. * Python: `pip install rerun-sdk` * Rust: `cargo add rerun` and `cargo install rerun-cli` * Online demo: <https://demo.rerun.io/version/0.8.0/> ### Overview & Highlights - `log_pinhole` is now easier to use in simple cases and supports non-RDF camera coordinates. [#2614](#2614) - You only need to set focal length and optional principal point instead of setting the full 3x3 matrix. - There is also a new argument: `camera_xyz` for setting the coordinate system. The default is RDF (the old default). This affects the visible camera frustum, how rays are projected when hovering a 2D image, and how depth clouds are projected. - The visualizer can now show coordinate arrows for all affine transforms within the view. [#2577](#2577) - Linestrips and oriented bounding boxes can now be logged via batch APIs in python. - See: `log_linestrips_2d`, `log_linestrips_3d`, [#2822](#2822) and `log_obbs` [#2823](#2823) - Rust users that build their own Viewer applications can now add fully custom Space Views. Find more information [here](https://www.rerun.io/docs/howto/extend-ui#custom-space-views-classes). - New optional `flush_timeout` specifies how long Rerun will wait if a TCP stream is disconnected during a flush. [#2821](#2821) - In Rust, `RecordingStream::connect` now requires `flush_timeout` specified as an `Option<Duration>`. - To keep default behavior, this can be specified using the `rerun::default_flush_time()` helper. - In Python `flush_init_sec` is now an optional argument to `rr.connect()` - In Rust, the `RecordingStream` now offers a stateful time API, similar to the Python APIs. [#2506](#2506) - You can now call `set_time_sequence`, `set_time_seconds`, and `set_time_nanos` directly on the `RecordingStream`, which will set the time for all subsequent logs using that stream. - This can be used as an alternative to the the previous `MsgSender::with_time` APIs. - The Rerun SDK now defaults to 8ms long microbatches instead of 50ms. This makes the default behavior more suitable for use-cases like real-time video feeds. [#2220](#2220) - Check out [the microbatching docs](https://www.rerun.io/docs/reference/sdk-micro-batching) for more information on fine-tuning the micro-batching behavior. - The web viewer now incremental loads `.rrd` files when streaming over HTTP. [#2412](#2412) ![Open Photogrammetry Preview](https://static.rerun.io/9fa26e73a197690e0403cd35f29e31c2941dea36_release_080_photogrammetry_full.png) ### Ongoing Refactors - There have been a number of significant internal changes going on during this release with little visible impact. This work will land across future releases, but is highlighted here since much of it is visible through the changelog. - The layout of the Viewer is now controlled by a Blueprint datastore. In the future this will allow for direct API control of the layout and configuration of the Viewer. A very early prototype of this functionality is available via the `rerun.experimental` module in Python. - An entirely new code-generation framework has been brought online for Rust, Python and C++. This will eventually enable new object-centric APIs with a more scalable, consistent, and ergonomic experience. - Bringup of C++ support is now underway and will eventually become our third officially supported SDK language. ### Known Regressions - Due to the Blueprint storage migration, blueprint persistence on web is currently broken. Will be resolved in: [#2579](#2579) ### In Detail #### 🐍 Python SDK - Clean up warnings printed when `rr.init` hasn't been called [#2209](#2209) - Normalise Python typing syntax to 3.8+ [#2361](#2361) - Simpler, sturdier stateful time tracking in both SDKs [#2506](#2506) - Fix not taking np.array for single colors [#2569](#2569) - Add a basic pyright config [#2610](#2610) - Improve `log_pinhole` and support non-RDF pinholes [#2614](#2614) - Expose batch APIs for linestrips [#2822](#2822) - Expose batch APIs for oriented bounding boxes [#2823](#2823) #### 🦀 Rust SDK - Add example for adding custom Space Views [#2328](#2328) - Simpler, sturdier stateful time tracking in both SDKs [#2506](#2506) - Automagic flush when `take()`ing a `MemorySinkStorage` [#2632](#2632) - Logging SDK: Log warnings if user data is dropped [#2630](#2630) - Add support for `RecordingStream::serve` [#2815](#2815) #### 🌁 Viewer Improvements - Better handle scroll-to-zoom in 3D views [#1764](#1764) - Add command to screenshot the application [#2293](#2293) - Show layout in blueprint tree view [#2465](#2465) - Double-click to select entity [#2504](#2504) - Add Rerun.io link/text in top bar [#2540](#2540) - New auto-layout of space views [#2558](#2558) - Add 'Dump datastore' command to palette [#2564](#2564) - Support any `dtype` for depth images [#2602](#2602) - Change "Save Selection" command to Cmd+Alt+S [#2631](#2631) - Consistent transform visualization for all entities with transforms [#2577](#2577) - Improve `log_pinhole` and support non-RDF pinholes [#2614](#2614) #### 🚀 Performance Improvements - Flush the batches every 8ms instead of 50 ms [#2220](#2220) - Replace `image` crate jpeg decoder with zune-jpeg [#2376](#2376) - Stream `.rrd` files when loading via http [#2412](#2412) #### 🪳 Bug Fixes - Fix deadlock when misusing the Caches [#2318](#2318) - Fix unstable order/flickering of "shown in" space view list on selection [#2327](#2327) - Fix transforms not applied to connections from transform context [#2407](#2407) - Fix texture clamping and color gradient selection being displayed incorrectly [#2394](#2394) - Fix projected ray length [#2482](#2482) - Tweak the depth bias multiplier for WebGL [#2491](#2491) - Clip image zoom rectangle [#2505](#2505) - Fix missing feature flags for benchmarks [#2515](#2515) - `run_all.py` script fixes [#2519](#2519) - Update egui_tiles with fix for drag-and-drop-panic [#2555](#2555) - Convert objectron proto.py back to using typing.List [#2559](#2559) - Exclude from `objectron/proto/objectron/proto.py` from `just py-format` [#2562](#2562) - Fix pinhole visualization not working with camera extrinsics & intrinsics on the same path [#2568](#2568) - Fix: always auto-layout spaceviews until the user interveens [#2583](#2583) - Fix freeze/crash when logging large times [#2588](#2588) - Update egui_tiles to fix crash [#2598](#2598) - Fix clicking object with single instance (of every component) selecting instance instead of entity [#2573](#2573) - Cleanup internal data-structures when process has been forked [#2676](#2676) - Fix shutdown race-condition by introducing a flush_timeout before dropping data [#2821](#2821) - Fix ui-scale based point/line sizes incorrectly scaled when zooming based on horizontal dimension [#2805](#2805) - Fix visibility toggle for maximized Space Views [#2806](#2806) - Fix loading file via CLI [#2807](#2807) - Fix disconnected space APIs in Python SDK [#2832](#2832) - Avoid unwrap when generating authkey [#2804](#2804) #### 🧑‍🏫 Examples - Add example template [#2392](#2392) - Show hidden url search param in `app.rerun.io` [#2455](#2455) - Minimal example of running an intel realsense depth sensor live [#2541](#2541) - Add a simple example to display Open Photogrammetry Format datasets [#2512](#2512) - Move `examples/api_demo` -> `tests/test_api` [#2585](#2585) #### 📚 Docs - Docs: link to `rr.save` and suggest `rerun` instead of `python -m rerun` [#2586](#2586) - Update docs about transforms [#2496](#2496) - Fixup remaining usages of log_rigid3 in docs [#2831](#2831) #### 🎨 Renderer Improvements - Expose type erased draw data that can be consumed directly [#2300](#2300) - Use less `mut` when using `RenderContext` [#2312](#2312) #### 🧑‍💻 Dev-experience - Better error messages in build.rs [#2173](#2173) - Recommend sccache in CONTRIBUTING.md [#2245](#2245) - introduce `re_tracing` [#2283](#2283) - lint: standardize formatting of let-else-return statements [#2297](#2297) - Centralized build tools in `re_build_tools` [#2331](#2331) - Lint for explicit quotes [#2332](#2332) - Added example screenshot instructions in `just upload --help` [#2454](#2454) - Added support for puling image from an URL to `upload_image.py` [#2462](#2462) - `setup_dev.sh` now installs pngcrush [#2470](#2470) - Added docs/code-examples to the directories checked by py-lint and py-format [#2476](#2476) - Link to demo in PR + check checkboxes [#2543](#2543) - Add script to find external issues we haven't commented on [#2532](#2532) - Move CI-related scripts to its own folder [#2561](#2561) - Render PR description as template [#2563](#2563) - Add basic testing automation against all version of Python using nox [#2536](#2536) - Run clippy on public API too [#2596](#2596) - Bump all `py-lint`-related package versions [#2600](#2600) - Crates publishing script [#2604](#2604) - Fix rust docs deploy [#2615](#2615) - Add support for .gitignore to scripts/lint.py [#2666](#2666) #### 🗣 Refactors - Refactor space-view dependencies: - Move spatial space view to its own crate [#2286](#2286) - Separate crate for bar chart space view [#2322](#2322) - Separate crate for time series space view [#2324](#2324) - Separate crate for tensor space view [#2334](#2334) - Separate viewport related files out to a new re_viewport crate [#2251](#2251) - Remove timepanel dependency from viewport [#2256](#2256) - New trait system for SpaceViews: - Initial Space View trait & port of text space views to the new Space View trait system [#2281](#2281) - Extend/iterate on SpaceViewClass framework with SceneContext & port SpatialSpaceView scene parts [#2304](#2304) - Finalize move of SpatialSpaceView to SpaceViewClass trait framework [#2311](#2311) - Typename cleanup in SpaceViewClass framework [#2321](#2321) - Automatic fallback for unrecognized Space View Class, start removing old ViewCategory [#2357](#2357) - Rename ScenePart -> ViewPartSystem + related renamings [#2674](#2674) - Dynamically registered space view (part/context) systems [#2688](#2688) - Viewer's command queue is now a channel, allowing to queue commands without mutable access [#2339](#2339) - Break up app.rs into parts [#2303](#2303) - Break out `re_log_types::component_types` as `re_components` [#2258](#2258) - Introduce StoreHub and rename Recording->Store [#2301](#2301) - Move StoreHub out of the Viewer during Update [#2330](#2330) - Expand CommandSender to support SystemCommand [#2344](#2344) - Use `camino` crate for UTF8 paths in `re_types_builder` [#2637](#2637) - Separate 2d & 3d spaceview classes, removal of `ViewCategory`, `SpaceViewClass` driven spawn heuristics [#2716](#2716) - Move object property heuristics to heuristics.rs [#2764](#2764) #### 📦 Dependencies - Version `rand` & friends at workspace level [#2508](#2508) - Update to PyO3 0.19 [#2350](#2350) - Pin `half` to `2.2.1` [#2587](#2587) #### 📘 Blueprint Changes - Drive blueprints off of a DataStore [#2010](#2010) - Split SpaceView -> SpaceViewState + SpaceViewBlueprint [#2188](#2188) - Split the Blueprint into AppBlueprint and ViewportBlueprint [#2358](#2358) - Swap the naming of Viewport and ViewportBlueprint [#2595](#2595) - Basic persistence for blueprints [#2578](#2578) #### 🏭 New Codegen Framework - Codegen/IDL 1: add more build tools [#2362](#2362) - Codegen/IDL 2: introduce `re_types_builder` [#2363](#2363) - Codegen/IDL 3: introduce `re_types` [#2369](#2369) - Codegen/IDL 4: definitions for a `Points2D` archetype [#2370](#2370) - Codegen/IDL 5: auto-generated Python code for `Points2D` [#2374](#2374) - Codegen/IDL 7: handwritten Python tests and extensions for `Points2D` [#2410](#2410) - Codegen/IDL 6: auto-generated Rust code for `Points2D` [#2375](#2375) - Codegen/IDL 8: handwritten Rust tests and extensions for `Points2D` [#2432](#2432) - Codegen'd Rust/Arrow 1: upgrading to actual `TokenStream`s [#2484](#2484) - Codegen'd Rust/Arrow 2: matching legacy definitions [#2485](#2485) - Codegen'd Rust/Arrow 3: misc fixes & improvements [#2487](#2487) - Codegen'd Rust/Arrow 4: out-of-sync definitions CI detection [#2545](#2545) - Codegen'd Rust/Arrow 5: doc, definitions and regression tests for combinatorial affixes [#2546](#2546) - Codegen'd Rust/Arrow 6: serialization [#2549](#2549) - Codegen'd Rust/Arrow 7: deserialization [#2554](#2554) - Codegen'd Rust/Arrow 8: carry extension metadata across transparency layers [#2570](#2570) - Codegen'd Rust/Arrow 9: Rust backport! [#2571](#2571) - End-to-end cross-language roundtrip tests for our archetypes [#2601](#2601) - Automatically derive `Debug` and `Clone` in Rust backend [#2613](#2613) - Generating (de)serialization code for dense unions in Rust backend [#2626](#2626) - Fix `FixedSizeList` deserialization edge-case + trivial optimizations [#2673](#2673) - Make `Datatype` & `Component` both inherit from `Loggable` [#2677](#2677) - Roundtrip-able `Transform3D`s [#2669](#2669) - Don't inline recursive datatypes in Rust backend [#2760](#2760) - Automatically derive `tuple_struct` attr and trivial `From` impls where possible [#2772](#2772) - Introduce roundtrip-able `Points3D` archetype (py + rs) [#2774](#2774) - Add `fmt::Debug` implementations to various types. [#2784](#2784) (thanks [@kpreid](https://github.com/kpreid)!) - Isolate testing types in Rust backend [#2810](#2810) - Fix out-of-sync codegen hash [#2567](#2567) - Python backport: add `log_any()` [#2581](#2581) - Integrate unit examples into codegen stack [#2590](#2590) - Disable codegen on windows [#2592](#2592) - Python codegen: big cleaning and paving the way towards transforms [#2603](#2603) - Automatically assume arrow transparency for components [#2608](#2608) - Fix wrong path being `rerun_if_changed()` in `compute_dir_hash` [#2612](#2612) - Support transparency at the semantic layer [#2611](#2611) - Don't use builtin `required` anymore, introduce `nullable` instead [#2619](#2619) - Rust codegen: generate proper docstrings [#2668](#2668) - Support nullable Arrow unions using virtual union arms [#2708](#2708) - Introduce support for querying Archetypes [#2743](#2743) - Introduce legacy shims and migrate DataCell to re_types::Component [#2752](#2752) #### 🌊 Starting work on C++ - Seed of C and C++ SDKs [#2594](#2594) - Move C++ SDK to own folder [#2624](#2624) - C++ codegen [#2678](#2678) - C++ codegen for reporting arrow data type for structs [#2756](#2756) - Don't inline recursive datatypes in C++ backend [#2765](#2765) - C++ codegen to_arrow_data_type for unions [#2766](#2766) - C++ codegen arrow serialize non-union components/datatypes without nested rerun types [#2820](#2820) - C++ codegen of structs and unions [#2707](#2707) - Fix cpp formatter differences [#2773](#2773) #### 🤷‍♂️ Other - test_api: set different app_id based on what test is run [#2599](#2599) - Introduce `rerun compare` to check whether 2 rrd files are functionally equivalent [#2597](#2597) - Remove `files.exclude` in vscode settings [#2621](#2621) - Support feature-gated rust attributes [#2813](#2813) ### Checklist * [x] I have read and agree to [Contributor Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md) * [x] I've included a screenshot or gif (if applicable) * [x] I have tested [demo.rerun.io](https://demo.rerun.io/pr/2842) (if applicable) - [PR Build Summary](https://build.rerun.io/pr/2842) - [Docs preview](https://rerun.io/preview/pr%3Arelease-0.8/docs) - [Examples preview](https://rerun.io/preview/pr%3Arelease-0.8/examples)

Wumpf and others added 30 commits June 28, 2023 14:48

remove somewhat redundant SpaceSpecs

81599ef

Camera frustums orient now correctly along view coordinates

a91c502

wip

don't cache view coordinates on space view state

881343b

remove unused unproject_as_ray method

eced0ca

fix camera ray and make image coordinate convention more explicit

49b3f1e

3D to 2D hover point projection works now also with weird view coordi…

ba27b00

…nates

use RUB/RDF constants and remove misleading comment about defaults. u…

c39606f

…ndo subtle change in default eye orientation

add test scene for view coordinates

f57a3f5

fix frustum ray casting not working on left handed coordinate systems

228dbbf

Merge remote-tracking branch 'origin/main' into andreas/fix-frustum-v…

816b4b2

…iew-coordinates

add arrows to the pinhole zoo

399c6d9

Merge branch 'main' into andreas/fix-frustum-view-coordinates

c52fe59

Set the correct coordinate system for the camera in the OPF example

d69c7ce

Import the future in the test

f72a057

Write out the left-handed coordinates we encounter

24d6527

Ignore left-handed coordinate system in test

6c535f2

Make sure we lint and format test code

3d33117

Link to #1387 in docs

c1bf004

Use up= argument for world coordinates

b8011f5

Merge branch 'main' into andreas/fix-frustum-view-coordinates

7fd44cf

Inline from_rub_quat

c80616f

Add camera_xyz=… argument to log_pinhole

1e48660

Only look for the view coordinates on the pinhole entity

c4bbc10

Better copy

4ece9fd

Add docstrings

931116e

Improve the documentation

119daaf

py-format

f84b73f

Merge branch 'main' into andreas/fix-frustum-view-coordinates

cca15ad

Test and handle depth cloud projection

dadfda8

py-format and py-lint

328a627

jleibs self-requested a review July 6, 2023 12:41

jleibs reviewed Jul 6, 2023

View reviewed changes

Merge branch 'main' into emilk/improve-pinhole

5fee0c0

jleibs approved these changes Jul 6, 2023

View reviewed changes

emilk and others added 13 commits July 7, 2023 15:35

improve comment

238ab3b

Co-authored-by: Jeremy Leibs <jeremy@rerun.io>

Add warning

c8ec5d9

Co-authored-by: Jeremy Leibs <jeremy@rerun.io>

Update source_hash.txt

8699522

Merge branch 'main' into emilk/improve-pinhole

7177111

Remove unnecessary camera_xyz in rgbd example

c96de33

be explicit

7aece38

Co-authored-by: Jeremy Leibs <jeremy@rerun.io>

Add try-catch blocks

3e2caa6

py-format

ff020e7

Better docstring

5f756f8

Make RDF the default, and add helper for converting between coordinates

62fad9c

Even better docstring

8cba3af

Merge branch 'main' into emilk/improve-pinhole

4cddaa5

Merge branch 'main' into emilk/improve-pinhole

280cf1a

emilk added 6 commits July 7, 2023 16:45

Improve docs even more

2075442

fix typos

ac29a55

FML

7d2c4e2

GOD DAMMIT THERE ARE TWO OF THEM

309c4c5

Merge branch 'main' into emilk/improve-pinhole

c65e61e

update source_hash.txt

a1314f7

emilk mentioned this pull request Jul 7, 2023

Automagic flush when take()ing a MemorySinkStorage #2632

Merged

3 tasks

emilk merged commit d4e05c5 into main Jul 7, 2023

emilk deleted the emilk/improve-pinhole branch July 7, 2023 15:48

Wumpf mentioned this pull request Jul 10, 2023

View frustum lines cross over image plane instead of going around #2660

Closed

jleibs mentioned this pull request Jul 26, 2023

Release 0.8 - Infrastructure investments and more transform improvements - 2023-07-27 #2842

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve `log_pinhole` and support non-RDF pinholes #2614

Improve `log_pinhole` and support non-RDF pinholes #2614

emilk commented Jul 5, 2023 •

edited by github-actions bot

Loading

jleibs Jul 6, 2023

emilk Jul 7, 2023

jleibs Jul 6, 2023

emilk Jul 7, 2023

emilk Jul 7, 2023

jleibs Jul 6, 2023

emilk Jul 7, 2023

jleibs Jul 6, 2023

emilk Jul 7, 2023

jleibs Jul 6, 2023

emilk Jul 7, 2023

jleibs Jul 6, 2023

jleibs Jul 6, 2023

emilk Jul 7, 2023

emilk commented Jul 7, 2023 •

edited

Loading

		/// Default right-handed view coordinates of `re_renderer`: X=Right, Y=Up, Z=Back.
		pub const RUB: Self = Self([ViewDir::Right, ViewDir::Up, ViewDir::Back]);

-        // View-coordinates are relevant here because without them we have no notion of what the image plane is.
-        // (it's not a given that e.g. XY is the camera image plane!)
-        // First transform to the "standard RUB" 3D camera and then from there to the image plane coordinate system.
+        // The pinhole view-coordinates are important here because they define how the image plane is aligned
+        // with the camera coordinate system. It is not a given that a user wants the image-plane aligned with the
+        // XY-plane in camera space.
+        //
+        // Because the [`Pinhole`] component currently assumes an input in the default `image_view_coordinates`
+        // we need to pre-transform the data from the user-defined `pinhole_view_coordiantes` to the required
+        // `image_view_coordinates`. We do this by converting the points into and out of RUB.
+        //
+        // TODO(...): When Pinhole is an archetype instead of a component, `pinhole.project` should do this
+        // internally.

Improve log_pinhole and support non-RDF pinholes #2614

Improve log_pinhole and support non-RDF pinholes #2614

Conversation

emilk commented Jul 5, 2023 • edited by github-actions bot Loading

What

Tests

Checklist

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

emilk commented Jul 7, 2023 • edited Loading

Improve `log_pinhole` and support non-RDF pinholes #2614

Improve `log_pinhole` and support non-RDF pinholes #2614

emilk commented Jul 5, 2023 •

edited by github-actions bot

Loading

emilk commented Jul 7, 2023 •

edited

Loading