How does Chat-Scene achieve understanding of the relative spatial relationships between objects without absolute coordinate inputs?

[zbqq]

In your implementation, the 3D features obtained by UNI3D and 2D features obtained by DINOV2 drop the absolute coordinate information of objects.

[ZzZZCHS]

Hi, thanks for your interest!

You’re correct that the current 3D/2D features lack explicit coordinate information. Only the DINOv2 features might capture some local spatial relationships within the images. The strong performance on datasets like ScanRefer could be attributed to their detailed attribute descriptions of the target object. The model can be further improved by incorporating encoders capable of capturing coordinate information.

[zbqq]

Thanks for your reply!

[zbqq]

By the way, when will the DINOv2 feature extraction code be released?

[ZzZZCHS]

Probably end of this month.

[cy94]

Hi @ZzZZCHS thanks for releasing your code and data. Any idea when you can release the DINOv2 feature extraction code?