question about train

[typhoonlee]

Hi，Thank you for your patient answer last time~
What type of GPU did you use during the training process? How many GPUs did you use? The efficiency is very slow during my training process.

[WeijingShi]

Hi @typhoonlee, the default train config uses two 11GB GPUs. A pair of GTX1080ti or a pair of RTX2080ti should be fine. And yes, the training takes around a week. If you have more GPUs, you can set the NUM_GPU in the train_config to a lager number. Note, the default batch size is 4. If you go beyond 4 GPUs, you will need to increase the batch size.

[typhoonlee]

Hello, I'm sorry to disturb you. What is the difference between run.py and eval.py and the offline package kitti_native_evaluation, and what is the relationship between them? What are their respective results? What should I do if I want to get experimental results for comparison with the paper?

[WeijingShi]

Hi @typhoonlee, the print out from run.py and eval.py are more or less internal/debug metrics related to this work. To compare with other works, kitti_native_evaluation is the right way to go.

Print out from run.py and eval.py: Accuracy and Recall are at point level, and car class is splited to front view car and side view car subclasses.

kitti_native_evaluation: Kitti 3D and BEV object detection scores (mAP), used in the paper.

Hope this helps. Thank you.

[typhoonlee]

Thank u very much！I got it
and i have another question：How to get this figure?

[WeijingShi]

Hi @typhoonlee, those figures in paper are drawn using OpenCV (boxes in the rgb) and Open3D (point cloud with colors). Check the visualization case VISUALIZATION_LEVEL == 2 in run.py for some examples. Thanks,

[typhoonlee]

Thank you very much, I got these pictures according to the method you said. But why is it different from the paper？

[typhoonlee]

And such an error statement will appear when the program is running, how can I solve this?

[WeijingShi]

Hi @typhoonlee, For the figures in the paper, we modified the visualization a bit to make it clean.

To remove the "lines", you just need to delete the graph_line_set in custom_draw_geometry_load_option() call.

To remove the color of the points or add custom color for each point, simply modify pcd.colors to your desired rgb color, e.g. setting all point to black pcd.colors = open3d.Vector3dVector(np.zeros((NUM_POINTS, 3))

Not quite sure about the warning, it seems related to pre-set the viewing angle though. If it does not break the code, it might be fine.

Sorry for the delayed reply. Thanks,

[typhoonlee]

Thank you very much~
and i have another question:
In the training , the vehicle and pedestrian categories are trained separately, and different pre-training weight files are obtained. Then in the inference phase, if I use checkpoints/car_auto_T3_train/, can I still detect traveling humans at the same time?

[abhigoku10]

@WeijingShi what does the multiple line from the bounding box signify!!

[WeijingShi]

Thank you very much~
and i have another question:
In the training , the vehicle and pedestrian categories are trained separately, and different pre-training weight files are obtained. Then in the inference phase, if I use checkpoints/car_auto_T3_train/, can I still detect traveling humans at the same time?

Hi @typhoonlee, unfortunately, the pre-trained model detects car and ped separately. Checkpoints/car_auto_T3_train only detects cars.

[WeijingShi]

@WeijingShi what does the multiple line from the bounding box signify!!

Hi @abhigoku10, the lines are the graph connections to the node which is outputting the box.

[typhoonlee]

Thank you very much~
and i have another question:
In the training , the vehicle and pedestrian categories are trained separately, and different pre-training weight files are obtained. Then in the inference phase, if I use checkpoints/car_auto_T3_train/, can I still detect traveling humans at the same time?

Hi @typhoonlee, unfortunately, the pre-trained model detects car and ped separately. Checkpoints/car_auto_T3_train only detects cars.

Thank you for your patience to answer.！Why is there a green box in the final prediction graph in the paper for cars, red for pedestrians, and blue for cyclists? How should I get the prediction results of all categories in a graph? Does it need to be retrained?

And...Why are there only car results and no pedestrians and cyclists in the results obtained by using kitti_native_evaluation, but there are still two types of 3DmAP results in the paper. How do you get them?

[typhoonlee]

Can you share the code for the result visualization? It will be very helpful to my study, thank you very much!

[WeijingShi]

Hi @typhoonlee, we provide ped_cyl_auto_T3_trainval checkpoint. You can test it in Kitti test split. Because the test split has no label. You need to submit the results to Kitti website to get the mAP number. If you need the accuracy on val split, you need to retrain it using train split only.

The qualitative visualization is the combined results of a car detection model and a ped-cyl model. The visualization code is drawn based on Open3D. You can select the points within the detection box and set their color in pcd.colors as desired.

The following function may be helpful:

Point-GNN/dataset/kitti_dataset.py

Line 143 in 48f3d79

def sel_xyz_in_box3d(label, xyz, expend_factor=(1.0, 1.0, 1.0)):

[typhoonlee]

Hi @typhoonlee, we provide ped_cyl_auto_T3_trainval checkpoint. You can test it in Kitti test split. Because the test split has no label. You need to submit the results to Kitti website to get the mAP number. If you need the accuracy on val split, you need to retrain it using train split only.

The qualitative visualization is the combined results of a car detection model and a ped-cyl model. The visualization code is drawn based on Open3D. You can select the points within the detection box and set their color in pcd.colors as desired.

The following function may be helpful:

Point-GNN/dataset/kitti_dataset.py

Line 143 in 48f3d79

def sel_xyz_in_box3d(label, xyz, expend_factor=(1.0, 1.0, 1.0)):

How do you get the visualization results after the two models are combined?
Oh,i see！
Is the corresponding .txt in the /eval/data file obtained by the two pre-training models combined, and then displayed as a whole?
If this is the case, how should run.py be modified? Looking forward to your reply！

[WeijingShi]

Hi @typhoonlee, If offline visualization is fine to you, you can just combine the results files, read them as label file for visualization:

Point-GNN/dataset/kitti_dataset.py

Line 703 in 48f3d79

def get_label(self, frame_idx, no_orientation=False):

Point-GNN/dataset/kitti_dataset.py

Line 1286 in 48f3d79

def vis_points(self, cam_points_in_img_with_rgb,

If you want an online combination, it's not supported by the current code. You need to rewrite run.py to load to models together and run them at the same time.