Multi_Device Box Estimation

[corkoh]

I would like to use the multi device box estimator. I have two Oak-D-Pro cameras. I have installed the requirements, printed out the checkerboard, updated the config file, and calibrated the two cameras. When I try to run the main.py for the box estimation, it gives me an error.

" if lwh > config.min_box_size:
TypeError: unsupported operand type(s) for *: 'float' and 'NoneType'"

I had it print l, w, and h to find that l, w, are returning 0.0 and h is 'none'

I dont see anywhere in the the get_dimensions function where its getting the height. I do see get_box_top returns height also. Could get_dimensions be resetting the height since its not called?

Or am I missing something?

[Erol444]

Hi @corkoh did you try to debug the error - where does the issue start?

[corkoh]

Here is the full error message. I looks like it starts in the PointCloudVisualizer class where it passes the camera list. The geometry:pointcloud is also a zero value.

I had tried to fill in the NoneType (h variable) by calling get_box_top before It does this math, and took out the return for h in the get_dimensions that is called right before this error.

That gave me some other errors, and so I figured I would reach out before I get carried away changing things I dont fully understand.

Traceback (most recent call last):
File "../depthai-experiments-master/gen2-multiple-devices/box-measurement/main.py", line 20, in
PointCloudVisualizer(cameras)
File "../depthai-experiments-master/gen2-multiple-devices/box-measurement/point_cloud_visualizer.py", line 33, in init
self.update()
File "../depthai-experiments-master/gen2-multiple-devices/box-measurement/point_cloud_visualizer.py", line 46, in update
self.vizualise_box()
File "../depthai-experiments-master/gen2-multiple-devices/box-measurement/point_cloud_visualizer.py", line 58, in vizualise_box
if lwh > config.min_box_size:
TypeError: unsupported operand type(s) for *: 'float' and 'NoneType'

[corkoh]

Update: I played with the RGBD-PointCloud-Fusion and noticed that with the calibration I had the point cloud never aligned. I have since done several calibrations and different camera angles until the pointcloud aligned almost perfectly.

I then tried the Box Measurement and It worked. The pointcloud was a lot smaller and it did not pick up the length or width of the box, only the height. The box was cut off in the smaller pointcloud, so that could be why I only got the height.

Are there specific heights, angles, etc to get better consistent results?

Also my end goal is checking the size of a mattress, would this be feasible?

[Erol444]

Hi @corkoh , Could you share some photos of camera angles, or how cameras see the box? wrt mattress - I'd say so, shouldn't be any more complicated than a box:)

[corkoh]

Here is similar to what I had 'working' earlier. (Top down view, I chose this because it also seemed the most accurate when doing the single device box estimation.) Its not working now, as I was trying to get a corner angle and a side angle to work with no success.

I can see in this with pointcloud fusion that it is not aligning the checkerboards on the correct Z axis. Camera1 checkerboard appears far below.

I can not consistently get the checkerboard to align on the pointcloud. I think this is my problem. Pressing the 'A' key does not align, it crashes every time. The one time It aligned almost correctly the one time the box estimation worked but did not capture all dimensions.

I have not had any success with any other camera angles yet.

Here is the output of the align button from the point cloud fusion:

Traceback (most recent call last):
File "../gen2-multiple-devices/rgbd-pointcloud-fusion/main.py", line 21, in
PointCloudVisualizer(cameras)
File "../gen2-multiple-devices/rgbd-pointcloud-fusion/point_cloud_visualizer.py", line 28, in init
self.update()
File "../gen2-multiple-devices/rgbd-pointcloud-fusion/point_cloud_visualizer.py", line 38, in update
self.point_cloud_window.poll_events()
File "../gen2-multiple-devices/rgbd-pointcloud-fusion/point_cloud_visualizer.py", line 13, in
self.point_cloud_window.register_key_callback(ord('A'), lambda vis: self.align_point_clouds())
File "../gen2-multiple-devices/rgbd-pointcloud-fusion/point_cloud_visualizer.py", line 60, in align_point_clouds
result_icp = o3d.pipelines.registration.registration_colored_icp(
AttributeError: module 'open3d' has no attribute 'pipelines'

[corkoh]

Ok I got box estimate to work again, the point cloud still does not line up perfectly. But the code is running since its close. I cleaned up my work space and got a proper table instead of a mattress. Not sure that changing out the surface helped any, but was able to get this angle working. The dimensions are not accurate. I imagine this is because the pointclouds dont fuse perfectly.

Here is my current set up:

Both cameras can see the checkerboard clearly, and when calibrating, they both find the same corner and orientate the x,y,z axis properly. So everything looks good when calibrating.

I rotated the pointcloud to show the discrepancies more, some of that floating away from the main piece is the checkerboard.