Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

he differences between HM3D-v1 and HM3D-v2 datasets for Object-Goal Navigation (ObjectNav) tasks. #2146

Open
yusirhhh opened this issue Jan 27, 2025 · 0 comments

Comments

@yusirhhh
Copy link

In the experiment, I evaluated the same model on both the HM3D-val v2 and HM3D-val v1 datasets. The results on the v2 dataset were significantly worse compared to those on the v1 dataset. Upon closer examination, I identified a potential issue: when the model predicts that it should stop, the distance to the goal consistently falls between 0.1m and 0.2m on the v2 dataset. Could this discrepancy arise from differences in dataset characteristics, annotation protocols, or inherent model biases when applied to the updated dataset version? I would appreciate your help in investigating this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant