Triangulation error with OpenCap data #142
Comments
|
Hi @sebastianhoefler, Thank you for pointing out the NaNs in the logs, you are right it was not clear and I updated the documentation: "If the original calibration file does not provide any residual errors, they will be logged as NaN. This is not an error and can be ignored." 2 cameras are the very minimum for any triangulation to be mathematically possible. But I do not think it is your problem, as we've been able to run challenging pose estimation with several people using only 2 cameras From what you are saying, I am pretty sure that the issue starts with calibration. I guess your iPhones are in portrait mode. when you run pose estimation, are the videos upright or sideways? If they are sideways, it means that the pose estimation algorithm does not detect the rotation metainformation of the video. In that case, first of all pose estimation results are not great, and then the triangulation stage will not be good since triangulation will not be coherent with calibration. Two things:
|
|
Hi @davidpagnon, thank you very much for getting back to me. Unfortunately, due to privacy concerns (this is a pre-trial and I don't have permission from the test subject), I cannot share the
When recording, the cameras (iPhone 13) were upright and not sideways. Data capture was done via opencapai. I tinkered a bit with the Click to expand log detailsINFO:root:INFO:root:Triangulation of 2D points for Klara_golf_pose2sim_test3, for all frames. 100%|██████████| 2365/2365 [00:11<00:00, 214.18it/s] INFO:root:All trc files have been converted to c3d. As you can see, the reprojection errors are very high and the consqequent steps don't yield any meaningful results and we get many This, in addition with the high mean reprojection error I mentioned above: makes me think that there might be some issue with calibration or with how the data was collected in the first place since reprojection errors this high are quite unusual. So either:
Would be interesting to hear your thoughts on the setup. Thank you again for your help! |
|
Hi again, Is there any way you could record yourself doing anything, even if it is just raising your hand, so that I can check the project? I expect your first assumption is the correct one. You can also install the Pose2Sim_Blender extension to see if your cameras and videos look like they are in the right position. Quick install is enough since you don't need to visualize OpenSim skeletons yet. N.B.: Just to let you know, you don't need to run personAssociation if you have only one person in the field of view. |
|
Hi David, I’ve received permission from one of the participants of the pre-trial to send the video data to you at contact@david-pagnon.com. Since this is a different participant, the reprojection errors are also somewhat different. However, I’m still encountering similar issues. For |
|
Hi, thank you for your data. It will remain confidential of course. The first thing that strikes me is that your video format is 360x640, while the calibration file is 720x1280. This is also the case in your original pickle files. Did you do anything to your videos, or set something up that could explain it in the OpenCap GUI? |
|
So I just checked, if you divide all the "matrix" values by 2 in your calibration file, it works fine. Now we need to figure out why the video resolution is divided by two. Please tell me if you figure out why! (in any case, unlike what I first thought, the portrait mode had nothing to do with your issue and it is correctly detected) |
|
Thank you! Great find! I didn't even realize it was scaled down by half. I don't think I am doing anything "wrong" in the opencap GUI. I simply download the data from the interface:
The downloaded folder looks like this: From the The I converted the |
|
Happy to learn it is solved! |
davidpagnon
changed the title
Pose2Sim.triangulation() Error: No persons have been triangulated
Hi @davidpagnon,
First, I’d like to thank you for this repo—it's exactly what I need for my project and has been incredibly helpful so far.
Unfortunately, I’m having an issue with the
Pose2Sim.triangulation()step.To provide some context, I’m working on capturing sports motions (golf swings, baseball swings, etc.) using OpenCap with a two-camera setup (two iPhones). For each camera (cam0 and cam1), I have a
cameraIntrinsicsExtrinsics.picklecalibration file and a corresponding video (golfswing_cam0_sync.mp4andgolfswing_cam1_sync.mp4). I want to run the pipeline up to and includingPose2Sim.kinematics().I already want to apologize for the lengthy message; I wanted to be thorough in providing all relevant details.
To begin, I followed the Using your own data section and set up the following folder structure:
I changed the
Config.tomlfile to handle opencap calibration data:convert_from = 'opencap' # 'caliscope', 'qualisys', 'optitrack', vicon', 'opencap', 'easymocap', 'biocv', 'anipose', or 'freemocap'Running
Pose2Sim.calibration()completes and I get:I was a bit skeptical at first since the camera calibration errors seem to be
[nan,nan]but I haven't been able to find the source of this error. However, since theCalib_opencap.tomlseems to get created, I wasn't sure if this is even an issue. TheCalib_opencap.tomllooks like this:Next, I ran
Pose2Sim.poseEstimation()which completes:Pose2Sim.synchronization()also works:Pose2Sim.personAssociation()also completes:Since I am also receiving
nanhere for reprojection error, I tried loweringreproj_error_threshold_associationto10inConfig.tomlwhich outputs:However, I am getting an error when trying to run
Pose2Sim.triangulation():Maybe I am missing something. I would really appreciate your help. Thank you very much!
The text was updated successfully, but these errors were encountered: