Implementation of the ACM MM 2023 paper: Semantic-based Selection Synthesis and Supervision for few-shot learning
Semantic-based Selection Synthesis and Supervision (4S) is a new method for few-shot learning, where semantics provide more diverse and informative supervision for recognizing novel objects. In this work, we firstly utilize semantic knowledge to explore the correlation of categories in the textual space and select base categories related to the given novel category, and then, we analyze the semantic knowledge to hallucinate the training samples by selectively synthesizing the contents from base and support samples (Distribution Exploration). Finally, we employ semantic knowledge as both soft and hard supervision to enrich the supervision for the fine-tuning procedure (Classifier Exploration). Empirical studies on four FSL benchmarks demonstrate the effectiveness of 4S.
Our implementation consists of 2 steps.
- download existing works with pre-trained backbone.
- extract base and novel features to your feature path.
python extract_feature.py
- modify the path to your feature path.
- run the Scripts_shot1.py & Scripts_shot5.py for 1-shot and 5-shot, respectively.
Specifically, for 1-shot,
python Scripts_shot1.py --feature_path your feature_path
for 5-shot,
python Scripts_shot5.py --feature_path your feature_path
for word embedding, please refer to my another page.
Any problems please contact me at jackie64321@gmail.com
@inproceedings{lu2023semantic,
title={Semantic-based Selection, Synthesis, and Supervision for Few-shot Learning},
author={Lu, Jinda and Wang, Shuo and Zhang, Xinyu and Hao, Yanbin and He, Xiangnan},
booktitle={Proceedings of the 31st ACM International Conference on Multimedia},
pages={3569--3578},
year={2023}
}