“look at” add-on to legacy Visualizer/ViewControl

[cwreynolds]

Just in case this might be useful to anyone else, here is a function to provide “look at” (look-from, look-at, up) functionality while using the legacy classes open3d::visualization::Visualizer and open3d::visualization::ViewControl.

I used this as a work around when I ran into problems using the newer open3d::visualization::visualizer::O3DVisualizer from c++ on macOS, see #7000

This function is similar (I think) to open3d::visualization::visualizer::O3DVisualizer::SetupCamera(fov, center, eye, up) but does not set fov. It alters a Visualizer’s ViewControl using the Convert[To/From]PinholeCameraParameters() API and a transform computed by open3d::visualization::gl_util::LookAt().

void setOpen3DVisualizerViewByFromAt(open3d::visualization::Visualizer& vis,
                                     const Eigen::Vector3d& look_from,
                                     const Eigen::Vector3d& look_at,
                                     const Eigen::Vector3d& up = {0, 1, 0})
{
    // Compute 4x4 from/at matrix.
    using namespace open3d::visualization::gl_util;
    Eigen::Matrix4d la_matrix = LookAt(look_from, look_at, up).cast<double>();

    // Get current PinholeCameraParameters.
    open3d::camera::PinholeCameraParameters pcp;
    vis.GetViewControl().ConvertToPinholeCameraParameters(pcp);

    // Overwrite the pcp's previous view matrix with the new look_at matrix.
    // (I am deeply puzzled by the need for that negation, but here we are.)
    pcp.extrinsic_ = -la_matrix;
    pcp.extrinsic_(3, 3) = 1;
    
    // Write back PinholeCameraParameters with new from/at view matrix.
    vis.GetViewControl().ConvertFromPinholeCameraParameters(pcp);
}

If you are still reading at this point, I am curious about that unexpected negation (pcp.extrinsic_ = -la_matrix). Can anyone explain how/why that is needed?

Finally as just a little context and eye-candy, this is how I am using it to provide a “follow camera” for a simulation of agents moving in a 3d environment. This is a boids model being used as a loss function in an optimization procedure acting on the flock parameters.

20241021.follow.cam.sample.mov