Skip to content

ACM MM 2023 paper: Semantic-based Selection Synthesis and Supervision for few-shot learning

Notifications You must be signed in to change notification settings

injadlu/4S-MM23-

Folders and files

NameName
Last commit message
Last commit date

Latest commit

51041fb · Nov 28, 2023

History

25 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

4S: Semantic-based Selection Synthesis and Supervision for few-shot learning

Implementation of the ACM MM 2023 paper: Semantic-based Selection Synthesis and Supervision for few-shot learning

Introduction

Semantic-based Selection Synthesis and Supervision (4S) is a new method for few-shot learning, where semantics provide more diverse and informative supervision for recognizing novel objects. In this work, we firstly utilize semantic knowledge to explore the correlation of categories in the textual space and select base categories related to the given novel category, and then, we analyze the semantic knowledge to hallucinate the training samples by selectively synthesizing the contents from base and support samples (Distribution Exploration). Finally, we employ semantic knowledge as both soft and hard supervision to enrich the supervision for the fine-tuning procedure (Classifier Exploration). Empirical studies on four FSL benchmarks demonstrate the effectiveness of 4S.

Get Started

Our implementation consists of 2 steps.

Extract feature

  1. download existing works with pre-trained backbone.
  2. extract base and novel features to your feature path.
python extract_feature.py

Run 1-shot and 5-shot

  1. modify the path to your feature path.
  2. run the Scripts_shot1.py & Scripts_shot5.py for 1-shot and 5-shot, respectively.
    Specifically, for 1-shot,
python Scripts_shot1.py --feature_path your feature_path

for 5-shot,

python Scripts_shot5.py --feature_path your feature_path

for word embedding, please refer to my another page.

An overview of our semantic-based selection, synthesis, and supervision method.

contact:

Any problems please contact me at jackie64321@gmail.com

References

@inproceedings{lu2023semantic,
  title={Semantic-based Selection, Synthesis, and Supervision for Few-shot Learning},
  author={Lu, Jinda and Wang, Shuo and Zhang, Xinyu and Hao, Yanbin and He, Xiangnan},
  booktitle={Proceedings of the 31st ACM International Conference on Multimedia},
  pages={3569--3578},
  year={2023}
}

About

ACM MM 2023 paper: Semantic-based Selection Synthesis and Supervision for few-shot learning

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages