-
Notifications
You must be signed in to change notification settings - Fork 73
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Waymo evaluation issue #29
Comments
@mincheoree please consider using
plus, I found in your log that the parsing ground truth is [0,7] [0]. No GT is loaded, and it is recommended that you check the loaded waymo gt_infos_train.pkl |
@BOBrown |
@mincheoree |
@BOBrown After moving to waymo dataset version 1.2, the issue is solved. |
@BOBrown @sky-fly97 I got another question, I trained SECOND model (without IOU branch) with 3% waymo dataset from scratch (without pretrained model) with provided yaml, waymo 3% dataset config, but got quite different result with your report in README.
While I get
Was your test sample interval set to 5? |
Hello,did you use VoxelBackBone8x as 3d backbone while using second? Note that we use VoxelResBackBone8x as the 3D backbone. Please change it at here. |
Hello, thanks for great works!
During Waymo dataset evaluation, I encountered 0 mAP and GT objects.
I searched in PCDet issue, the author says that Waymo 1.0.0 is incompatible with evaluation, and they use Waymo 1.2.0 dataset as default, referring to this: open-mmlab/OpenPCDet#1102
However, in your AD-PT paper, you mentioned that you used Waymo 1.0.0 as your experiment setting. As your codebase is based on PCDet, may I ask if you also encountered this 0 AP error during waymo evaluation? Is Waymo 1.0.0 okay for evaluation?
The text was updated successfully, but these errors were encountered: