Skip to content

Entity-Aware and Motion-Aware Transformers for Language-driven Action Localization(IJCAI-22)

Notifications You must be signed in to change notification settings

shuoyang129/eamat

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Entity-Aware and Motion-Aware Transformers for Language-driven Action

This is implementation for the paper "Entity-Aware and Motion-Aware Transformers for Language-driven Action" (IJCAI 2022)

# preparing environment
bash conda.sh

Dataset Preparation

We use VSLNet's data. The visual features can be download here, for CharadesSTA we use the "new" fold, and for TACoS we use the "old" fold, annotation and other details can be found here and then modify the line 81~91 of "dataset/BaseDataset.py" to your own path.

Quick Start

Train

python main.py --cfg experiments/charades/EAMAT.yaml --mode train
python main.py --cfg experiments/tacos/EAMAT.yaml --mode train

a new fold "results" are created.

Citation

If you feel this project helpful to your research, please cite our work.

@inproceedings{DBLP:conf/ijcai/YangW22,
  author    = {Shuo Yang and
               Xinxiao Wu},
  title     = {Entity-aware and Motion-aware Transformers for Language-driven Action
               Localization},
  booktitle = {Proceedings of the Thirty-First International Joint Conference on
               Artificial Intelligence, {IJCAI} 2022, Vienna, Austria, 23-29 July
               2022},
  pages     = {1552--1558},
  publisher = {ijcai.org},
  year      = {2022},
  url       = {https://doi.org/10.24963/ijcai.2022/216},
  doi       = {10.24963/ijcai.2022/216},
  timestamp = {Wed, 27 Jul 2022 16:43:00 +0200},
  biburl    = {https://dblp.org/rec/conf/ijcai/YangW22.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}

About

Entity-Aware and Motion-Aware Transformers for Language-driven Action Localization(IJCAI-22)

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published