Reproduction of paper results

[alexbgl]

Hi @leolyj,

thank you for sharing code and data of your very interesting work!

When trying to reproduce the results from your paper, I encountered the following two issues:

1. for GTA-SF ---> KITTI-SF, I managed to roughly reproduce your results, both when just doing inference with your provided checkpoints and when training the adaptation beginning from the checkpoint of the baseline model. Thus, both data pre-processing and training routine seem to work here. However, I noticed that training and validation data from the KITTI dataset are identical. Is that correct or did I miss something?

2. for GTA-SF ---> Lyft-SF, I get roughly the same results as in the paper when doing inference with your provided checkpoints. Thus, data pre-processing and seems to be correct here as well. However, when starting the adaptation process from the checkpoint of the baseline model on my own, I could not reach the performance reported in the paper. Compared to an EPE3D of 0.1277, I achieve around 0.135 before the model converges/begins to slightly overfit (please see the attached training curve). I also noted that the model achieved top performance already after around 2800 iterations while you trained for around 46.000 iterations (15 epochs * 12430 data samples / 4 batch size). Do you have any idea what the reasons could be? Are there possibly any hyper-parameters I need to adjust?

Thanks in advance!

[JZ-9962]

Hi, have you modified the EPS parameter when adapting to different target dataset? In our experiments, EPS is set to 0.6 for waymo, 0.4 for kitti and 0.3 for lyft. You can try to adjust this hyper-parameter. Good luck!