Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RSDK-1258] World State to Point Cloud Conversion Method #1675

Merged
merged 110 commits into from
Dec 30, 2022

Conversation

nfranczak
Copy link
Member

@nfranczak nfranczak commented Dec 16, 2022

This ticket involves adding a conversion function for box, sphere, and point geometry types into a PointCloud
The conversion is two stepped:

  1. call the ToPoints method on the given geometry. ToPoints returns type []r3.Vector
  2. call VectorsToPointCloud. This method has one argument of type []r3.Vector[]. VectorsToPointCloud returns a PointCloud

This ticket also removes the Pose field from the Point struct and replaces it with r3.Vector

@viambot viambot added the safe to test This pull request is marked safe to test from a trusted zone label Dec 16, 2022
@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Dec 16, 2022
@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Dec 16, 2022
@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Dec 16, 2022
myMap["xIter"] = 0.5
myMap["yIter"] = 0.5
myMap["zIter"] = 0.5
box.ToPointCloud(myMap)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll need to think about how to test this better, because as is this will not catch any changes that would break your function

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

based off the dimensions we can calculate how many points the r3.vector list contains. because we also have the center and rotation matrix we can apply that to the existing vertices and check within the r3.vector list for each corresponding vertex

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I would like to see as a test for this, would look like, say, a 2x2x2 box, a 1x1.5x4 box, and a radius=2.5 sphere, at hardcoded poses, with the expected set of points also hardcoded.

@@ -352,3 +355,84 @@ func separatingAxisTest(positionDelta, plane r3.Vector, halfSizeA, halfSizeB [3]
}
return sum
}

// TODO: add function description
func (b *box) ToPointCloud(options map[string]interface{}) ([]r3.Vector, error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this can just become resolution int since it should be the same in all dimensions

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

float64, not int. Probably in units of points per mm?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, good catch Peter

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok sure, but we need to keep map[string]interface{} because this allows for empty inputs, i.e. the user can use default spacing if they don't want to thing about it


var faces [][]float64
// which faces are these
for i := 0.0; i <= dims[0]; i += options["xIter"].(float64) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

starting at the center, adding 2 points per iteration, per face, and working inside to outsid iterating until i <= halfSize will mean that we'll run into symmetric errors if the resolution does not evenly divide the dimension

Copy link
Member Author

@nfranczak nfranczak Dec 16, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed, but if we allow the user to dictate the resolution should it be on them to figure this out?
What do you recommend?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also suppose we want to render a cube with side length 1.
both 1 / 0.0.3 and 1 / 0.13 would result in symmetric errors with the former being much less noticeable.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

}
}
// which faces are these
for j := 0.0; j <= dims[1]; j += options["yIter"].(float64) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should probably be a helper function

Copy link
Member Author

@nfranczak nfranczak Dec 16, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed.
edit: now that i am thinking about it a helper function for these three nested for loops could be a little tricky. This is because the inside of all three does different things. I think that it is still possible to have them be a helper function with the use of switch statements or something of that manner.

}
}
// what does this do
min := b.Vertices()[0]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't we offset by the box.Pose().Point() instead of going through this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep


// TODO: add descriptions
func (p *point) ToPointCloud(options map[string]interface{}) ([]r3.Vector, error) {
myVec := r3.Vector{p.pose.Point().X, p.pose.Point().Y, p.pose.Point().Z}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return []r3.Vector{p.pose.Point()} should work here. Also I'm noticing I really shouldn't have made Point store a Pose since theres no point in storing orientation data 🙃

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should I edit the struct of Point to remove Pose?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would actually be a good fix too thanks

spatialmath/sphere.go Show resolved Hide resolved
}
}
// what does this do
for _, v := range faces {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that both this, and the matrix transformation below, can be replaced by transforming each point by PoseInverse(box.Pose())

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that this will work but this part of the code will be gone in the newest version - pushing now

@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Dec 16, 2022
@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Dec 29, 2022
@nfranczak nfranczak changed the title Data 857: World State to Point Cloud Conversion Method RSDK-1258: World State to Point Cloud Conversion Method Dec 29, 2022
@nfranczak nfranczak changed the title RSDK-1258: World State to Point Cloud Conversion Method [RSDK-1258] World State to Point Cloud Conversion Method Dec 29, 2022
// the boolean values which are passed into the fillFaces method allow for common edges to be
// ignored. This removes duplicate points on the box's surface
var facePoints [][]r3.Vector
facePoints = append(facePoints, fillFaces(b.halfSize, iter, 0, true, false))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

facePoints should be a []r3.Vector, and these appends should use elipses ... to expand the arguments for append.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Example usage:
https://stackoverflow.com/questions/40657553/all-uses-of-ellipsis

Here's some more background on variadic functions such as append:
https://programming.guide/go/three-dots-ellipsis.html

@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Dec 29, 2022
@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Dec 29, 2022
@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Dec 29, 2022
@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Dec 30, 2022
spatialmath/geometry.go Outdated Show resolved Hide resolved
Copy link
Member

@raybjork raybjork left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Copy link
Member

@biotinker biotinker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Nice work!

@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Dec 30, 2022
@github-actions
Copy link
Contributor

Code Coverage

Note: merge base coverage results not available, comparing against closest f361208~2=(6ab9820) instead

Package Line Rate Delta Health
go.viam.com/rdk/components/arm 56% 0.00%
go.viam.com/rdk/components/arm/universalrobots 13% 0.00%
go.viam.com/rdk/components/arm/xarm 2% 0.00%
go.viam.com/rdk/components/arm/yahboom 7% 0.00%
go.viam.com/rdk/components/audioinput 55% +0.34%
go.viam.com/rdk/components/base 63% 0.00%
go.viam.com/rdk/components/base/agilex 62% 0.00%
go.viam.com/rdk/components/base/boat 41% 0.00%
go.viam.com/rdk/components/base/wheeled 76% 0.00%
go.viam.com/rdk/components/board 68% 0.00%
go.viam.com/rdk/components/board/arduino 10% 0.00%
go.viam.com/rdk/components/board/commonsysfs 47% 0.00%
go.viam.com/rdk/components/board/fake 38% 0.00%
go.viam.com/rdk/components/board/numato 19% 0.00%
go.viam.com/rdk/components/board/pi 50% 0.00%
go.viam.com/rdk/components/camera 65% 0.00%
go.viam.com/rdk/components/camera/align 63% -0.48%
go.viam.com/rdk/components/camera/fake 67% 0.00%
go.viam.com/rdk/components/camera/ffmpeg 77% +1.43%
go.viam.com/rdk/components/camera/transformpipeline 77% 0.00%
go.viam.com/rdk/components/camera/videosource 49% -0.63%
go.viam.com/rdk/components/encoder/fake 77% 0.00%
go.viam.com/rdk/components/gantry 60% 0.00%
go.viam.com/rdk/components/gantry/multiaxis 84% 0.00%
go.viam.com/rdk/components/gantry/oneaxis 86% 0.00%
go.viam.com/rdk/components/generic 83% 0.00%
go.viam.com/rdk/components/gripper 68% 0.00%
go.viam.com/rdk/components/input 87% 0.00%
go.viam.com/rdk/components/input/gpio 84% 0.00%
go.viam.com/rdk/components/motor 77% 0.00%
go.viam.com/rdk/components/motor/dimensionengineering 63% 0.00%
go.viam.com/rdk/components/motor/dmc4000 69% 0.00%
go.viam.com/rdk/components/motor/fake 57% 0.00%
go.viam.com/rdk/components/motor/gpio 64% 0.00%
go.viam.com/rdk/components/motor/gpiostepper 57% +0.59%
go.viam.com/rdk/components/motor/tmcstepper 62% 0.00%
go.viam.com/rdk/components/movementsensor 75% 0.00%
go.viam.com/rdk/components/movementsensor/cameramono 40% 0.00%
go.viam.com/rdk/components/movementsensor/gpsnmea 37% 0.00%
go.viam.com/rdk/components/movementsensor/gpsrtk 28% 0.00%
go.viam.com/rdk/components/posetracker 86% 0.00%
go.viam.com/rdk/components/sensor 86% 0.00%
go.viam.com/rdk/components/sensor/ultrasonic 34% 0.00%
go.viam.com/rdk/components/servo 68% 0.00%
go.viam.com/rdk/components/servo/gpio 71% 0.00%
go.viam.com/rdk/config 76% 0.00%
go.viam.com/rdk/control 57% 0.00%
go.viam.com/rdk/data 79% 0.00%
go.viam.com/rdk/examples/customresources/demos/complexmodule 0% 0.00%
go.viam.com/rdk/examples/customresources/demos/remoteserver 0% 0.00%
go.viam.com/rdk/grpc 25% 0.00%
go.viam.com/rdk/ml 67% 0.00%
go.viam.com/rdk/ml/inference 70% 0.00%
go.viam.com/rdk/module 64% 0.00%
go.viam.com/rdk/module/modmanager 79% 0.00%
go.viam.com/rdk/motionplan 59% -3.14%
go.viam.com/rdk/octree 98% 0.00%
go.viam.com/rdk/operation 82% 0.00%
go.viam.com/rdk/pointcloud 72% +0.06%
go.viam.com/rdk/protoutils 59% 0.00%
go.viam.com/rdk/referenceframe 70% 0.00%
go.viam.com/rdk/registry 89% 0.00%
go.viam.com/rdk/resource 84% 0.00%
go.viam.com/rdk/rimage 78% 0.00%
go.viam.com/rdk/rimage/depthadapter 94% 0.00%
go.viam.com/rdk/rimage/transform 73% 0.00%
go.viam.com/rdk/rimage/transform/cmd/extrinsic_calibration 67% 0.00%
go.viam.com/rdk/robot 85% -0.95%
go.viam.com/rdk/robot/client 82% 0.00%
go.viam.com/rdk/robot/framesystem 68% 0.00%
go.viam.com/rdk/robot/impl 81% 0.00%
go.viam.com/rdk/robot/server 59% 0.00%
go.viam.com/rdk/robot/web 62% 0.00%
go.viam.com/rdk/robot/web/stream 87% 0.00%
go.viam.com/rdk/services/armremotecontrol 71% 0.00%
go.viam.com/rdk/services/armremotecontrol/builtin 52% 0.00%
go.viam.com/rdk/services/baseremotecontrol 71% 0.00%
go.viam.com/rdk/services/baseremotecontrol/builtin 79% 0.00%
go.viam.com/rdk/services/datamanager 79% 0.00%
go.viam.com/rdk/services/datamanager/builtin 78% 0.00%
go.viam.com/rdk/services/datamanager/datacapture 70% 0.00%
go.viam.com/rdk/services/datamanager/datasync 0% 0.00%
go.viam.com/rdk/services/motion 63% 0.00%
go.viam.com/rdk/services/motion/builtin 88% 0.00%
go.viam.com/rdk/services/navigation 53% 0.00%
go.viam.com/rdk/services/sensors 77% 0.00%
go.viam.com/rdk/services/sensors/builtin 97% 0.00%
go.viam.com/rdk/services/shell 14% 0.00%
go.viam.com/rdk/services/slam 84% 0.00%
go.viam.com/rdk/services/slam/builtin 72% 0.00%
go.viam.com/rdk/services/vision 80% 0.00%
go.viam.com/rdk/services/vision/builtin 74% 0.00%
go.viam.com/rdk/session 97% 0.00%
go.viam.com/rdk/spatialmath 84% +1.49%
go.viam.com/rdk/subtype 92% 0.00%
go.viam.com/rdk/utils 72% 0.00%
go.viam.com/rdk/vision 26% 0.00%
go.viam.com/rdk/vision/chess 80% 0.00%
go.viam.com/rdk/vision/delaunay 87% 0.00%
go.viam.com/rdk/vision/keypoints 92% 0.00%
go.viam.com/rdk/vision/objectdetection 76% -6.94%
go.viam.com/rdk/vision/odometry 60% 0.00%
go.viam.com/rdk/vision/odometry/cmd 0% 0.00%
go.viam.com/rdk/vision/segmentation 49% 0.00%
go.viam.com/rdk/web/server 25% -0.50%
Summary 65% (20638 / 31579) -0.20%

@nfranczak nfranczak merged commit 6a3508a into viamrobotics:main Dec 30, 2022
@nfranczak nfranczak deleted the DATA-857 branch December 30, 2022 19:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
safe to test This pull request is marked safe to test from a trusted zone
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants