Data list for quantitative comparison

[Byeongjun1022]

Hello, thank you for sharing your great research and releasing the code.

I would like to ask whether the test set used for the quantitative comparison on the UnrealStereo4K dataset is 'splits/test_in.txt' or 'splits/test_out.txt'.
Additionally, could you please clarify which test set was used for the quantitative comparison on the CityScapes dataset?

Thank you!

[zhyever]

Hi, thanks for your interest in our work.

We use val.txt to evaluate the model as default following previous metric depth papers. The numbers and in-domain results in our paper are all from this set. The text_in.txt indicates in-domain test set, which could also be good for evaluation but we just left it here. The text_out.txt is a set for out-of-domain evaluation. The model performs much worse here, indicating some generalization issues waiting for solutions.

For the CityScapes, the training set includes the standard training data plus the extra set. The validation set is also just the official one. I put the corresponding files right below. Note that the evaluation protocal presented in this paper is slightly different from the original paper. As we claimed, the current one makes much more sense and we encourage researchers to report the updated results in their paper for fair comparisons. Please check here for more information.

train.txt
val.txt

Btw, I just noticed some small issues in the cityscapes training docs. I'm working on it and I will also add the specific number for comparison for the cistyscapes. I will leave a message here after finishing it.

[Byeongjun1022]

Thanks for fast reply.
It is really helpful.