Skip to content

knowledge-computing/multimodal-text-linking

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

11 Commits
 
 
 
 
 
 
 
 

Repository files navigation

LIGHT: Multi-Modal Text Linking for Historical Maps

This repository provides training and inference scripts for LIGHT, an approach for linking text instances on historical maps.

LIGHT is a multi-modal learning model that jointly models text content, spatial layouts, polygonal shapes, and visual features to resolve complex linking challenges in scanned historical maps.

📢 The paper has been accepted for oral presentation at ICDAR 2025.
📄 View on arXiv

📚 Pretraining

1. Polygon Encoder Pretraining

CUDA_VISIBLE_DEVICES="0" torchrun \
  --nproc_per_node=1 \
  --nnodes=1 \
  --node_rank=0 \
  --master_addr=127.0.0.1 \
  --master_port=14476 \
  pretrain_poly.py --config configs/pretrain_poly.yaml

2. Full LIGHT Model Pretraining

CUDA_VISIBLE_DEVICES="0" torchrun \
  --nproc_per_node=1 \
  --nnodes=1 \
  --node_rank=0 \
  --master_addr=127.0.0.1 \
  --master_port=14476 \
  pretrain.py --config configs/pretrain_light.yaml

🔧 Fine-Tuning

python train.py --config configs/light.yaml

🔍 Inference

python inference.py \
  --test_dataset MapText_test \
  --out_file predict.json \
  --model_dir ./_weights/finetune_light \
  --anno_path icdar24-test-png-annotations.json \
  --img_dir icdar24-test-png/test_images/

📁 Notes

🔗 References

If you find this repository useful in your own work, we would appreciate a citation to the accompanying paper:

@inproceedings{weinman2024counting,
   authors = {Lin, Yijun and Olson, Rhett and Wu, Junhan and Chiang, Yao-Yi and Weinman, Jerod},
   title = {LIGHT: Multi-Modal Text Linking on Historical Maps},
   booktitle = {19th International Conference on Document Analysis and Recognition ({ICDAR} 2025)},
   series = {Lecture Notes in Computer Science},
   publisher = {Springer},
   location = {Wuhan, China},
   year = {2025}
}

About

Repo for ICDAR2025 paper "LIGHT: Multi-Modal Text Linking on Historical Maps"

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 2

  •  
  •