Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RSDK-1035] Camera Properties call fails #1802

Merged
merged 4 commits into from
Jan 25, 2023

Conversation

bazile-clyde
Copy link
Contributor

@bazile-clyde bazile-clyde commented Jan 23, 2023

Bug fix: See script in RSDK-1035 for steps to reproduce.

Comment on lines +169 to +177
if intrinsics := resp.IntrinsicParameters; intrinsics != nil {
result.IntrinsicParams = &transform.PinholeCameraIntrinsics{
Width: int(intrinsics.WidthPx),
Height: int(intrinsics.HeightPx),
Fx: intrinsics.FocalXPx,
Fy: intrinsics.FocalYPx,
Ppx: intrinsics.CenterXPx,
Ppy: intrinsics.CenterYPx,
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was the bug fix for the original issue. Here, if resp.IntrinsicParameters == nil we would panic with the error

invalid memory address or nil pointer dereference

on line 171.

Checking if the resp.IntrinsicParameters == nil before attempting to access its methods fixes the issue.

@@ -237,6 +237,104 @@ func TestClient(t *testing.T) {
})
}

func TestClientProperties(t *testing.T) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This just tests we've fixed the bug and checks for similar issues.

@@ -204,3 +206,17 @@ func TestGetCameraMatrix(t *testing.T) {
test.That(t, intrinsicsK.At(1, 2), test.ShouldEqual, intrinsics.Ppy)
test.That(t, intrinsicsK.At(2, 2), test.ShouldEqual, 1)
}

func TestNilIntrinsics(t *testing.T) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a user gets back nil intrinsic parameters (which is now possible thanks to my fix 😞) and attempts to call any of the methods below we'd panic. For example

props, err := cam.Properties(context.Background())
if err != nil {
	logger.Fatal("attempting to get cam properties: %v", err)
	return
}
props.GetCameraMatrix() // panic!

This is easily fixable by checking if the parameters are nil in these methods. It's also idiomatic to do this in Go AFAIK.

I noticed for the distortion parameters we have a struct transform.NoDistortion that solves the same problem but I'm not sure what the advantage would be if we adopted that solution for intrinsic parameters too. Seems like they're both valid but I could be missing something. @bhaney it looks like you created that struct so if you have an opinion on this let me know.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, I think using nil as the "no intrinsics" and "no distortion" scenario is much more intuitive than creating a specific struct. Because then you have to remember to always use the "no such thing" struct, and nil is still there potentially causing problems!

@viambot viambot added the safe to test This pull request is marked safe to test from a trusted zone label Jan 24, 2023
@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Jan 24, 2023
@bazile-clyde
Copy link
Contributor Author

@bhaney removed NoDistortion

Copy link
Member

@bhaney bhaney left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have just one fix for the panic test, otherwise LGTM!

Comment on lines -9 to -10
// NoneDistortionType applies no distortion to an input image. Essentially an identity transform.
NoneDistortionType = DistortionType("no_distortion")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for getting rid of this, making "no distortion" just be nil is much simpler

@@ -204,3 +206,17 @@ func TestGetCameraMatrix(t *testing.T) {
test.That(t, intrinsicsK.At(1, 2), test.ShouldEqual, intrinsics.Ppy)
test.That(t, intrinsicsK.At(2, 2), test.ShouldEqual, 1)
}

func TestNilIntrinsics(t *testing.T) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, I think using nil as the "no intrinsics" and "no distortion" scenario is much more intuitive than creating a specific struct. Because then you have to remember to always use the "no such thing" struct, and nil is still there potentially causing problems!

Comment on lines 215 to 221
nilIntrinsics.CheckValid()
nilIntrinsics.GetCameraMatrix()
nilIntrinsics.PixelToPoint(0.0, 0.0, 0.0)
nilIntrinsics.PointToPixel(0.0, 0.0, 0.0)
nilIntrinsics.ImagePointTo3DPoint(image.Point{}, rimage.Depth(0))
nilIntrinsics.RGBDToPointCloud(&rimage.Image{}, &rimage.DepthMap{})
nilIntrinsics.PointCloudToRGBD(pointcloud.PointCloud(nil))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can use test.ShouldNotPanic so that it doesn't actually panic and cause the testing program to crash

You need to wrap them each in a function i.e.

test.That(t, func() {  nilIntrinsics.CheckValid() }, test.ShouldNotPanic)

Copy link
Contributor Author

@bazile-clyde bazile-clyde Jan 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, fancy! Done.

@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Jan 24, 2023
@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Jan 25, 2023
Copy link
Member

@gvaradarajan gvaradarajan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, sorry for the delay!

@github-actions
Copy link
Contributor

Code Coverage

Note: merge base coverage results not available, comparing against closest aa1914d~2=(9e2f63d) instead

Package Line Rate Delta Health
go.viam.com/rdk/components/arm 56% 0.00%
go.viam.com/rdk/components/arm/universalrobots 41% 0.00%
go.viam.com/rdk/components/arm/xarm 2% 0.00%
go.viam.com/rdk/components/arm/yahboom 7% 0.00%
go.viam.com/rdk/components/audioinput 55% 0.00%
go.viam.com/rdk/components/base 63% 0.00%
go.viam.com/rdk/components/base/agilex 62% 0.00%
go.viam.com/rdk/components/base/boat 41% 0.00%
go.viam.com/rdk/components/base/wheeled 65% 0.00%
go.viam.com/rdk/components/board 66% 0.00%
go.viam.com/rdk/components/board/arduino 10% 0.00%
go.viam.com/rdk/components/board/commonsysfs 48% 0.00%
go.viam.com/rdk/components/board/fake 38% 0.00%
go.viam.com/rdk/components/board/numato 19% 0.00%
go.viam.com/rdk/components/board/pi 50% 0.00%
go.viam.com/rdk/components/camera 65% +1.52%
go.viam.com/rdk/components/camera/align 64% 0.00%
go.viam.com/rdk/components/camera/fake 64% 0.00%
go.viam.com/rdk/components/camera/ffmpeg 76% 0.00%
go.viam.com/rdk/components/camera/transformpipeline 78% -0.62%
go.viam.com/rdk/components/camera/videosource 45% -0.38%
go.viam.com/rdk/components/encoder 4% 0.00%
go.viam.com/rdk/components/encoder/fake 76% 0.00%
go.viam.com/rdk/components/gantry 60% 0.00%
go.viam.com/rdk/components/gantry/multiaxis 84% 0.00%
go.viam.com/rdk/components/gantry/oneaxis 86% 0.00%
go.viam.com/rdk/components/generic 83% 0.00%
go.viam.com/rdk/components/gripper 68% 0.00%
go.viam.com/rdk/components/input 87% 0.00%
go.viam.com/rdk/components/input/gpio 84% 0.00%
go.viam.com/rdk/components/motor 77% 0.00%
go.viam.com/rdk/components/motor/dimensionengineering 63% 0.00%
go.viam.com/rdk/components/motor/dmc4000 69% 0.00%
go.viam.com/rdk/components/motor/fake 57% 0.00%
go.viam.com/rdk/components/motor/gpio 64% 0.00%
go.viam.com/rdk/components/motor/gpiostepper 56% 0.00%
go.viam.com/rdk/components/motor/tmcstepper 62% 0.00%
go.viam.com/rdk/components/movementsensor 75% 0.00%
go.viam.com/rdk/components/movementsensor/cameramono 46% 0.00%
go.viam.com/rdk/components/movementsensor/gpsnmea 36% 0.00%
go.viam.com/rdk/components/movementsensor/gpsrtk 28% 0.00%
go.viam.com/rdk/components/posetracker 86% 0.00%
go.viam.com/rdk/components/sensor 86% 0.00%
go.viam.com/rdk/components/sensor/ultrasonic 26% 0.00%
go.viam.com/rdk/components/servo 68% 0.00%
go.viam.com/rdk/components/servo/gpio 71% 0.00%
go.viam.com/rdk/config 75% 0.00%
go.viam.com/rdk/control 57% 0.00%
go.viam.com/rdk/data 79% 0.00%
go.viam.com/rdk/examples/customresources/demos/complexmodule 0% 0.00%
go.viam.com/rdk/examples/customresources/demos/remoteserver 0% 0.00%
go.viam.com/rdk/grpc 25% 0.00%
go.viam.com/rdk/ml 67% 0.00%
go.viam.com/rdk/ml/inference 70% 0.00%
go.viam.com/rdk/module 63% 0.00%
go.viam.com/rdk/module/modmanager 79% 0.00%
go.viam.com/rdk/motionplan 59% -8.65%
go.viam.com/rdk/octree 98% 0.00%
go.viam.com/rdk/operation 82% 0.00%
go.viam.com/rdk/pointcloud 72% 0.00%
go.viam.com/rdk/protoutils 59% 0.00%
go.viam.com/rdk/referenceframe 71% 0.00%
go.viam.com/rdk/registry 89% 0.00%
go.viam.com/rdk/resource 84% 0.00%
go.viam.com/rdk/rimage 78% 0.00%
go.viam.com/rdk/rimage/depthadapter 94% 0.00%
go.viam.com/rdk/rimage/transform 75% +0.67%
go.viam.com/rdk/rimage/transform/cmd/extrinsic_calibration 67% 0.00%
go.viam.com/rdk/robot 85% 0.00%
go.viam.com/rdk/robot/client 81% 0.00%
go.viam.com/rdk/robot/framesystem 65% 0.00%
go.viam.com/rdk/robot/impl 80% -0.22%
go.viam.com/rdk/robot/server 55% -0.59%
go.viam.com/rdk/robot/web 62% 0.00%
go.viam.com/rdk/robot/web/stream 87% 0.00%
go.viam.com/rdk/services/armremotecontrol 71% 0.00%
go.viam.com/rdk/services/armremotecontrol/builtin 52% 0.00%
go.viam.com/rdk/services/baseremotecontrol 71% 0.00%
go.viam.com/rdk/services/baseremotecontrol/builtin 79% 0.00%
go.viam.com/rdk/services/datamanager 79% 0.00%
go.viam.com/rdk/services/datamanager/builtin 78% 0.00%
go.viam.com/rdk/services/datamanager/datacapture 70% 0.00%
go.viam.com/rdk/services/datamanager/datasync 0% 0.00%
go.viam.com/rdk/services/motion 63% 0.00%
go.viam.com/rdk/services/motion/builtin 88% 0.00%
go.viam.com/rdk/services/navigation 53% 0.00%
go.viam.com/rdk/services/sensors 77% 0.00%
go.viam.com/rdk/services/sensors/builtin 97% 0.00%
go.viam.com/rdk/services/shell 14% 0.00%
go.viam.com/rdk/services/slam 84% 0.00%
go.viam.com/rdk/services/slam/builtin 70% -0.06%
go.viam.com/rdk/services/vision 80% 0.00%
go.viam.com/rdk/services/vision/builtin 74% 0.00%
go.viam.com/rdk/session 97% 0.00%
go.viam.com/rdk/spatialmath 82% 0.00%
go.viam.com/rdk/subtype 92% 0.00%
go.viam.com/rdk/utils 72% +0.18%
go.viam.com/rdk/vision 26% 0.00%
go.viam.com/rdk/vision/chess 80% 0.00%
go.viam.com/rdk/vision/delaunay 87% 0.00%
go.viam.com/rdk/vision/keypoints 92% 0.00%
go.viam.com/rdk/vision/objectdetection 82% 0.00%
go.viam.com/rdk/vision/odometry 60% 0.00%
go.viam.com/rdk/vision/odometry/cmd 0% 0.00%
go.viam.com/rdk/vision/segmentation 49% 0.00%
go.viam.com/rdk/web/server 25% 0.00%
Summary 65% (21309 / 32937) -0.49%

@bazile-clyde bazile-clyde merged commit 73ac48f into viamrobotics:main Jan 25, 2023
@bazile-clyde bazile-clyde deleted the RSDK-1035 branch January 25, 2023 22:11
randhid pushed a commit to randhid/rdk that referenced this pull request Feb 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
safe to test This pull request is marked safe to test from a trusted zone
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants