How to run the model in a N-way K-shot manner?

[kamilc]

Hi,

Thank you so much for an amazing paper and for publishing this code!

I have a question regarding the way to use one of your trained models in a N-way K-shot manner. I can see the repository closely follows detectron2. This means that I can load the model's weights and get predictions via DefaultPredictor. That class's __call__ only takes an input image though. I'm struggling to see the way to incorporate the N*K embeddings (and ways to get them in the first place) of the support set, and then to use the predictor to get the results. (I've downloaded the weights for the mask_rcnn_R_50_FPN_ft_fullclsag_cos_bh_all_5shot_manyiters_V2_correct config).

I'm very likely not seeing something very obvious, apologies for asking naive questions in this case. I'll appreciate any pointers on ways to use it in the N-way K-shot manner though!

Thank you so much again

[Lily19970207]

Hello,did you sove the problem?

I trained the model step by step:

1. configs/coco-experiments/mask_rcnn_R_50_FPN_fc_fullclsag_base.yaml
2. configs/coco-experiments/mask_rcnn_R_50_FPN_ft_fullclsag_cos_bh_base.yaml
3. configs/coco-experiments/mask_rcnn_R_50_FPN_fullclsag_metric_avg_cos_bh_normalized_all_10shot.yaml（trained on 20 novel classes）

but I get the result:
{
"bbox": {
"AP": 0.6086382577102198,
"AP50": 0.9686674621591191,
"AP75": 0.651878918279263,
"APs": 0.16141502940083804,
"APm": 0.397410775472805,
"APl": 0.8720534612187981,
"AP-person": 0.4201384547180321,
"AP-bicycle": 0.036003600360036005,
"AP-car": 6.707690074690357,
"AP-motorcycle": 0.23102310231023102,
"AP-airplane": 12.85165286472409,
"AP-bus": 18.615992723296102,
"AP-train": 9.025457589149143,
"AP-truck": 0.575665427910193,
"AP-boat": 0.001288515813115742,
"AP-traffic light": 0.0,
"AP-fire hydrant": 0.0,
"AP-stop sign": 0.0,
"AP-parking meter": 0.0,
"AP-bench": 0.00584188632494989,
"AP-bird": 0.0,
"AP-cat": 0.0,
"AP-dog": 0.220306377521335,
"AP-horse": 0.0,
"AP-sheep": 0.0,
"AP-cow": 0.0,
"AP-elephant": 0.0,
"AP-bear": 0.0,
"AP-zebra": 0.0,
"AP-giraffe": 0.0,
"AP-backpack": 0.0,
"AP-umbrella": 0.0,
"AP-handbag": 0.0,
"AP-tie": 0.0,
"AP-suitcase": 0.0,
"AP-frisbee": 0.0,
"AP-skis": 0.0,
"AP-snowboard": 0.0,
"AP-sports ball": 0.0,
"AP-kite": 0.0,
"AP-baseball bat": 0.0,
"AP-baseball glove": 0.0,
"AP-skateboard": 0.0,
"AP-surfboard": 0.0,
"AP-tennis racket": 0.0,
"AP-bottle": 0.0,
"AP-wine glass": 0.0,
"AP-cup": 0.0,
"AP-fork": 0.0,
"AP-knife": 0.0,
"AP-spoon": 0.0,
"AP-bowl": 0.0,
"AP-banana": 0.0,
"AP-apple": 0.0,
"AP-sandwich": 0.0,
"AP-orange": 0.0,
"AP-broccoli": 0.0,
"AP-carrot": 0.0,
"AP-hot dog": 0.0,
"AP-pizza": 0.0,
"AP-donut": 0.0,
"AP-cake": 0.0,
"AP-chair": 0.0,
"AP-couch": 0.0,
"AP-potted plant": 0.0,
"AP-bed": 0.0,
"AP-dining table": 0.0,
"AP-toilet": 0.0,
"AP-tv": 0.0,
"AP-laptop": 0.0,
"AP-mouse": 0.0,
"AP-remote": 0.0,
"AP-keyboard": 0.0,
"AP-cell phone": 0.0,
"AP-microwave": 0.0,
"AP-oven": 0.0,
"AP-toaster": 0.0,
"AP-sink": 0.0,
"AP-refrigerator": 0.0,
"AP-book": 0.0,
"AP-clock": 0.0,
"AP-vase": 0.0,
"AP-scissors": 0.0,
"AP-teddy bear": 0.0,
"AP-hair drier": 0.0,
"AP-toothbrush": 0.0
},
"segm": {
"AP": 0.591411294434657,
"AP50": 0.9301556758380742,
"AP75": 0.6180622877437737,
"APs": 0.07606327093023611,
"APm": 0.359037156413112,
"APl": 0.9040012941146236,
"AP-person": 0.0,
"AP-bicycle": 0.0,
"AP-car": 6.47596948525144,
"AP-motorcycle": 0.18151815181518152,
"AP-airplane": 9.893301675508773,
"AP-bus": 19.9174939425781,
"AP-train": 10.238609475394998,
"AP-truck": 0.5967282715647985,
"AP-boat": 0.0015103648841640705,
"AP-traffic light": 0.0,
"AP-fire hydrant": 0.0,
"AP-stop sign": 0.0,
"AP-parking meter": 0.0,
"AP-bench": 0.004966239164042062,
"AP-bird": 0.0,
"AP-cat": 0.0,
"AP-dog": 0.0028059486110554375,
"AP-horse": 0.0,
"AP-sheep": 0.0,
"AP-cow": 0.0,
"AP-elephant": 0.0,
"AP-bear": 0.0,
"AP-zebra": 0.0,
"AP-giraffe": 0.0,
"AP-backpack": 0.0,
"AP-umbrella": 0.0,
"AP-handbag": 0.0,
"AP-tie": 0.0,
"AP-suitcase": 0.0,
"AP-frisbee": 0.0,
"AP-skis": 0.0,
"AP-snowboard": 0.0,
"AP-sports ball": 0.0,
"AP-kite": 0.0,
"AP-baseball bat": 0.0,
"AP-baseball glove": 0.0,
"AP-skateboard": 0.0,
"AP-surfboard": 0.0,
"AP-tennis racket": 0.0,
"AP-bottle": 0.0,
"AP-wine glass": 0.0,
"AP-cup": 0.0,
"AP-fork": 0.0,
"AP-knife": 0.0,
"AP-spoon": 0.0,
"AP-bowl": 0.0,
"AP-banana": 0.0,
"AP-apple": 0.0,
"AP-sandwich": 0.0,
"AP-orange": 0.0,
"AP-broccoli": 0.0,
"AP-carrot": 0.0,
"AP-hot dog": 0.0,
"AP-pizza": 0.0,
"AP-donut": 0.0,
"AP-cake": 0.0,
"AP-chair": 0.0,
"AP-couch": 0.0,
"AP-potted plant": 0.0,
"AP-bed": 0.0,
"AP-dining table": 0.0,
"AP-toilet": 0.0,
"AP-tv": 0.0,
"AP-laptop": 0.0,
"AP-mouse": 0.0,
"AP-remote": 0.0,
"AP-keyboard": 0.0,
"AP-cell phone": 0.0,
"AP-microwave": 0.0,
"AP-oven": 0.0,
"AP-toaster": 0.0,
"AP-sink": 0.0,
"AP-refrigerator": 0.0,
"AP-book": 0.0,
"AP-clock": 0.0,
"AP-vase": 0.0,
"AP-scissors": 0.0,
"AP-teddy bear": 0.0,
"AP-hair drier": 0.0,
"AP-toothbrush": 0.0
}
}

I don't know how to train the last step?

[Lily19970207]

How to do the last step?

python tools/run_train.py --config-file path_to_config_file --method pure-metric-averaged

[zhangyuxuan321]

How to do the last step?

python tools/run_train.py --config-file path_to_config_file --method pure-metric-averaged

Hi, do you solve this problem? I have the same problem with you.