OSMLoc: Single Image-Based Visual Localization in OpenStreetMap with Semantic and Geometric Guidances

This is the official PyTorch implementation of the following publication:

OSMLoc: Single Image-Based Visual Localization in OpenStreetMap with Semantic and Geometric Guidances
Youqi Liao*,Xieyuanli Chen*,Shuhao Kang Jianping Li**, Zhen Dong, Hongchao Fan, Bisheng Yang
Technical Report
Paper | Arxiv | Project-page | Video

🔭 Introduction

TL;DR: OSMLoc is an image-to-OpenstreetMap (I2O) visual localization framework with geometric and semantic guidance.

Abstract: OpenStreetMap (OSM), an online and versatile source of volunteered geographic information (VGI), is widely used for human self-localization by matching nearby visual observations with vectorized map data. However, due to the divergence in modalities and views, image-to-OSM (I2O) matching and localization remain challenging for robots, preventing the full utilization of VGI data in the unmanned ground vehicles and logistic industry. Inspired by the fact that the human brain relies on different regions when processing geometric and semantic information for spatial localization tasks, in this paper, we propose the OSMLoc. OSMLoc is a brain-inspired monocular visual localization method with semantic and geometric guidance to improve accuracy, robustness, and generalization ability. First, we equip the OSMLoc with the visual foundational model to extract powerful image features. Second, a geometry-guided depth distribution adapter is proposed to bridge the monocular depth estimation and camera-to-BEV transform. Thirdly, the semantic embeddings from the OSM data are utilized as auxiliary guidance for image-to-OSM feature matching. To validate the proposed OSMLoc, we collect a worldwide cross-area and cross-condition (CC) benchmark for extensive evaluation. Experiments on the MGL dataset, CC validation benchmark, and KITTI dataset have demonstrated the superiority of our method.

🆕 News

• 2024-11-20: Project page (with introduction video) is available!🎉
• 2024-11-20: The code, pre-trained models, and validation benchmark will be available upon acceptance of the paper.

💡 Citation

If you find this repo helpful, please give us a star~.Please consider citing OSMLoc if this program benefits your project.

@article{liao2024osmloc,
  title={OSMLoc: Single Image-Based Visual Localization in OpenStreetMap with Geometric and Semantic Guidances},
  author={Liao, Youqi and Chen, Xieyuanli and Kang, Shuhao and Li, Jianping and Dong, Zhen and Fan, Hongchao and Yang, Bisheng},
  journal={arXiv preprint arXiv:2411.08665},
  year={2024}
}

🔗 Related Projects

We sincerely thank the excellent projects:

• OrienterNet for pioneering I2O visual localization approach;
• Retrieval for extensive evaluation;
• Refine for extensive evaluation;
• Range-MCL for Monto Carlo localization framework;
• Freereg for excellent template;