Second coarse calibration (with a matrix in place) goes wrong

[cwidisgroup]

Doing a cwipc_register --coarse when there is already a non-unity matrix does not work: apparently the coordinates are interpreted as if in camera space.

Happens for both Realsense and Kinect:

First Realsense calibration:

cwipc_register: Use coarse alignment class MultiCameraCoarseArucoRgb
cwipc_register: camera 0: show RGB image with detected marker
cwipc_register: camera 0: _find_markers: Auruco-IDs: [0], 2D-Arease: [[[289.0, 276.0], [293.0, 242.0], [312.0, 248.0], [308.0, 279.0]]]
cwipc_register: camera 0: find_markers: marker 0, corner 0: u,v,d=(289, 276, 2063)
cwipc_register: camera 0: find_markers: marker 0, corner 0: 3d-point=(-0.7066450715065002, 0.17218974232673645, 2.062999963760376)
cwipc_register: camera 0: find_markers: marker 0, corner 1: u,v,d=(293, 242, 2096)
cwipc_register: camera 0: find_markers: marker 0, corner 1: 3d-point=(-0.6978207230567932, 0.006894778460264206, 2.0959999561309814)
cwipc_register: camera 0: find_markers: marker 0, corner 2: u,v,d=(312, 248, 2255)
cwipc_register: camera 0: find_markers: marker 0, corner 2: 3d-point=(-0.6491027474403381, 0.0392642617225647, 2.255000114440918)
cwipc_register: camera 0: find_markers: marker 0, corner 3: u,v,d=(308, 279, 2247)
cwipc_register: camera 0: find_markers: marker 0, corner 3: 3d-point=(-0.6683283448219299, 0.20320859551429749, 2.246999979019165)
cwipc_register: find_markers_all_tiles: camera 0: marker ids: dict_keys([0])
cwipc_register: find_markers_all_tiles: camera 0: marker 0:  3D corner: (-0.7066450715065002, 0.17218974232673645, 2.062999963760376), distance to next: 0.168
78771786371255
cwipc_register: find_markers_all_tiles: camera 0: marker 0:  3D corner: (-0.6978207230567932, 0.006894778460264206, 2.0959999561309814), distance to next: 0.1
6941745759342142
cwipc_register: find_markers_all_tiles: camera 0: marker 0:  3D corner: (-0.6491027474403381, 0.0392642617225647, 2.255000114440918), distance to next: 0.1652
615210624075
cwipc_register: find_markers_all_tiles: camera 0: marker 0:  3D corner: (-0.6683283448219299, 0.20320859551429749, 2.246999979019165), distance to next: 0.190
48975408700106
cwipc_register: coarse: attempting to register tiles [0]
cwipc_register: _align_marker: camera 0: rmse error=2.282277603727453
cwipc_register: coarse: camera 1 cameramask 0: marker 0: created transformation matrix
coarse aligner ran for 15.144 seconds
cwipc_register: save cameraconfig.json
cwipc_register: reload cameraconfig.json

Second Realsense calibration (with existing matrix):

cwipc_register: Use coarse alignment class MultiCameraCoarseArucoRgb
cwipc_register: camera 0: show RGB image with detected marker
cwipc_register: camera 0: _find_markers: Auruco-IDs: [0], 2D-Arease: [[[289.0, 276.0], [293.0, 242.0], [312.0, 248.0], [308.0, 279.0]]]
cwipc_register: camera 0: find_markers: marker 0, corner 0: u,v,d=(289, 276, 2070)
cwipc_register: camera 0: find_markers: marker 0, corner 0: 3d-point=(0.0824352353811264, 5.149960270500742e-05, 0.08737196773290634)
cwipc_register: camera 0: find_markers: marker 0, corner 1: u,v,d=(293, 242, 2096)
cwipc_register: camera 0: find_markers: marker 0, corner 1: 3d-point=(-0.08578772097826004, -0.004118451848626137, 0.08614612370729446)
cwipc_register: camera 0: find_markers: marker 0, corner 2: u,v,d=(312, 248, 2263)
cwipc_register: camera 0: find_markers: marker 0, corner 2: 3d-point=(-0.08145361393690109, -0.00013376466813497245, -0.09010807424783707)
cwipc_register: camera 0: find_markers: marker 0, corner 3: u,v,d=(308, 279, 2236)
cwipc_register: camera 0: find_markers: marker 0, corner 3: 3d-point=(0.08410055190324783, 0.0017894268967211246, -0.0868455171585083)
cwipc_register: find_markers_all_tiles: camera 0: marker ids: dict_keys([0])
cwipc_register: find_markers_all_tiles: camera 0: marker 0:  3D corner: (0.0824352353811264, 5.149960270500742e-05, 0.08737196773290634), distance to next: 0.
1682790962507632
cwipc_register: find_markers_all_tiles: camera 0: marker 0:  3D corner: (-0.08578772097826004, -0.004118451848626137, 0.08614612370729446), distance to next:
0.17635250072675188
cwipc_register: find_markers_all_tiles: camera 0: marker 0:  3D corner: (-0.08145361393690109, -0.00013376466813497245, -0.09010807424783707), distance to nex
t: 0.16559747815586986
cwipc_register: find_markers_all_tiles: camera 0: marker 0:  3D corner: (0.08410055190324783, 0.0017894268967211246, -0.0868455171585083), distance to next: 0
.17423411179297935
cwipc_register: coarse: attempting to register tiles [0]
cwipc_register: _align_marker: camera 0: rmse error=0.004801343268258395
cwipc_register: coarse: camera 1 cameramask 0: marker 0: created transformation matrix
coarse aligner ran for 7.047 seconds
cwipc_register: save cameraconfig.json
cwipc_register: reload cameraconfig.json

First Kinect calibration:

cwipc_register: Use coarse alignment class MultiCameraCoarseArucoRgb
cwipc_register: camera 0: show RGB image with detected marker
cwipc_register: camera 0: _find_markers: Auruco-IDs: [0], 2D-Arease: [[[397.0, 414.0], [403.0, 363.0], [433.0, 372.0], [427.0, 419.0]]]
cwipc_register: camera 0: find_markers: marker 0, corner 0: u,v,d=(397, 414, 2019)
xxxjack cwipc_kinect aux-op map2d3d
cwipc_register: camera 0: find_markers: marker 0, corner 0: 3d-point=(-0.7841528654098511, 0.14410357177257538, 2.0190000534057617)
cwipc_register: camera 0: find_markers: marker 0, corner 1: u,v,d=(403, 363, 2045)
xxxjack cwipc_kinect aux-op map2d3d
cwipc_register: camera 0: find_markers: marker 0, corner 1: 3d-point=(-0.7752408981323242, -0.022854313254356384, 2.0450000762939453)
cwipc_register: camera 0: find_markers: marker 0, corner 2: u,v,d=(433, 372, 2206)
xxxjack cwipc_kinect aux-op map2d3d
cwipc_register: camera 0: find_markers: marker 0, corner 2: 3d-point=(-0.7311125993728638, 0.007607860490679741, 2.2060000896453857)
cwipc_register: camera 0: find_markers: marker 0, corner 3: u,v,d=(427, 419, 2177)
xxxjack cwipc_kinect aux-op map2d3d
cwipc_register: camera 0: find_markers: marker 0, corner 3: 3d-point=(-0.7418206334114075, 0.17351974546909332, 2.177000045776367)
cwipc_register: find_markers_all_tiles: camera 0: marker ids: dict_keys([0])
cwipc_register: find_markers_all_tiles: camera 0: marker 0:  3D corner: (-0.7841528654098511, 0.14410357177257538, 2.0190000534057617), distance to next: 0.16
920508184923982
cwipc_register: find_markers_all_tiles: camera 0: marker 0:  3D corner: (-0.7752408981323242, -0.022854313254356384, 2.0450000762939453), distance to next: 0.
16969459354923763
cwipc_register: find_markers_all_tiles: camera 0: marker 0:  3D corner: (-0.7311125993728638, 0.007607860490679741, 2.2060000896453857), distance to next: 0.1
687673490769644
cwipc_register: find_markers_all_tiles: camera 0: marker 0:  3D corner: (-0.7418206334114075, 0.17351974546909332, 2.177000045776367), distance to next: 0.166
1966507786037
cwipc_register: coarse: attempting to register tiles [0]
cwipc_register: _align_marker: camera 0: rmse error=2.2547614505735463
cwipc_register: coarse: camera 1 cameramask 0: marker 0: created transformation matrix
coarse aligner ran for 3.653 seconds
cwipc_register: save cameraconfig.json
cwipc_register: reload cameraconfig.json

Second Kinect calibration (with matrix in place):

cwipc_register: Use coarse alignment class MultiCameraCoarseArucoRgb
cwipc_register: camera 0: show RGB image with detected marker
cwipc_register: camera 0: _find_markers: Auruco-IDs: [0], 2D-Arease: [[[397.0, 414.0], [403.0, 363.0], [433.0, 372.0], [427.0, 419.0]]]
cwipc_register: camera 0: find_markers: marker 0, corner 0: u,v,d=(397, 414, 2018)
xxxjack cwipc_kinect aux-op map2d3d
cwipc_register: camera 0: find_markers: marker 0, corner 0: 3d-point=(0.08441109955310822, 0.0008761166245676577, 0.08370459824800491)
cwipc_register: camera 0: find_markers: marker 0, corner 1: u,v,d=(403, 363, 2043)
xxxjack cwipc_kinect aux-op map2d3d
cwipc_register: camera 0: find_markers: marker 0, corner 1: 3d-point=(-0.08454210311174393, 0.0010207240702584386, 0.0868036225438118)
cwipc_register: camera 0: find_markers: marker 0, corner 2: u,v,d=(433, 372, 2205)
xxxjack cwipc_kinect aux-op map2d3d
cwipc_register: camera 0: find_markers: marker 0, corner 2: 3d-point=(-0.0839821994304657, 0.0008221166208386421, -0.08372444659471512)
cwipc_register: camera 0: find_markers: marker 0, corner 3: u,v,d=(427, 419, 2177)
xxxjack cwipc_kinect aux-op map2d3d
cwipc_register: camera 0: find_markers: marker 0, corner 3: 3d-point=(0.08463718742132187, -0.0002386039705015719, -0.08335395902395248)
cwipc_register: find_markers_all_tiles: camera 0: marker ids: dict_keys([0])
cwipc_register: find_markers_all_tiles: camera 0: marker 0:  3D corner: (0.08441109955310822, 0.0008761166245676577, 0.08370459824800491), distance to next: 0
.1689816840773282
cwipc_register: find_markers_all_tiles: camera 0: marker 0:  3D corner: (-0.08454210311174393, 0.0010207240702584386, 0.0868036225438118), distance to next: 0
.17052910397103918
cwipc_register: find_markers_all_tiles: camera 0: marker 0:  3D corner: (-0.0839821994304657, 0.0008221166208386421, -0.08372444659471512), distance to next:
0.16862313012005728
cwipc_register: find_markers_all_tiles: camera 0: marker 0:  3D corner: (0.08463718742132187, -0.0002386039705015719, -0.08335395902395248), distance to next:
 0.16706242927575637
cwipc_register: coarse: attempting to register tiles [0]
cwipc_register: _align_marker: camera 0: rmse error=0.004027934752430377
cwipc_register: coarse: camera 1 cameramask 0: marker 0: created transformation matrix
coarse aligner ran for 3.381 seconds
cwipc_register: save cameraconfig.json
cwipc_register: reload cameraconfig.json