[segmentation] Update angle between 2 planes to be the smallest angle between them

[benedikt-mayer-inmach]

Fix angle calculation as communicated in #3842

[larshg]

I'm just curious, but if you have a plane with normal 0,0,1 and a plane with normal 0,0,-1 you would say the angle between the planes are 0?

I would think it would be 180 degrees? Even though the planes are parallel the "top surface" would be in either direction?

[benedikt-mayer-inmach]

I guess generally speaking that indeed is up to interpretation.
In my mind a plane with normal 0,0,1 also has the normal 0,0,-1.

Now there might be cases where you can define sensible ways to decide for "top" or "bottom surface". This could then be used for a clustering algorithm.

In the case of our concrete example the normals are estimated by use of PCA of the covariance matrix (https://github.com/PointCloudLibrary/pcl/blob/master/examples/segmentation/example_extract_clusters_normals.cpp). The resulting direction of the normal in this case is ambiguous.
From https://pcl-tutorials.readthedocs.io/en/latest/normal_estimation.html#normal-estimation → "In general, because there is no mathematical way to solve for the sign of the normal, its orientation computed via Principal Component Analysis (PCA) as shown above is ambiguous, and not consistently oriented over an entire point cloud dataset."
Therefore this approach doesn't seem possible/sensible here.

[SergioRAgostinho]

I'm just curious, but if you have a plane with normal 0,0,1 and a plane with normal 0,0,-1 you would say the angle between the planes are 0?

Yes, because the homogeneous equation which defines a plane contemplates this ambiguity.

I would think it would be 180 degrees? Even though the planes are parallel the "top surface" would be in either direction?

This is a concept which is important in computer graphics and meshes but when handling point clouds, the default case is that the normal direction is intrinsically ambiguous.

[larshg]

Thanks for thoughts and explanations :)