Skip to content

Latest commit

 

History

History
executable file
·
2533 lines (2287 loc) · 90.8 KB

MODEL_ZOO.md

File metadata and controls

executable file
·
2533 lines (2287 loc) · 90.8 KB
Raw

Model Zoo and Baselines

We provide a set of benchmark results and pre-trained models available for download. On PASCAL VOC and COCO, we provide results for seed 0, which are consistent with results from previous works. We also provide results over multiple seeds.

Available Models

We provide several models that are discussed in our paper.

  • Base Model: model after first stage of base training.
  • FRCN+ft-full: fine-tuning the entire model until convergence.
  • TFA w/ fc: our approach with a FC-based classifier.
  • TFA w/ cos: our approach with a cosine similarity based classifier.

Loading Models in Code

You can access the models from code using the fsdet.model_zoo API (official documentation here).

  1. Pick a model and its config file from model zoo, for example, COCO-detection/faster_rcnn_R_101_FPN_ft_all_1shot.yaml.
  2. from fsdet import model_zoo
model = model_zoo.get(
   "COCO-detection/faster_rcnn_R_101_FPN_ft_all_1shot.yaml", trained=True)

PASCAL VOC Object Detection Baselines

Results on Seed 0

Base Models

Name Split
bAP
bAP50
bAP75		 download
Base Model 1 54.9 80.8 61.1 model | metrics
Base Model 2 55.1 81.9 61.4 model | metrics
Base Model 3 55.6 82.0 61.5 model | metrics

1-shot

Name Split
AP
AP50
AP75
bAP
bAP50
bAP75
nAP
nAP50
nAP75		 download
FRCN+ft-full 1 34.0 55.4 36.7 42.2 68.9 45.6 9.3 15.2 10.0 model | metrics
TFA w/ fc 1 43.8 69.3 47.6 51.6 80.2 56.4 20.3 36.8 21.0 model | metrics
TFA w/ cos 1 44.0 69.7 47.8 50.9 79.6 55.6 23.4 39.8 24.5 model | metrics
FRCN+ft-full 2 30.1 50.1 31.9 38.1 62.4 40.9 6.0 13.4 4.8 model | metrics
TFA w/ fc 2 41.2 64.7 45.3 51.8 80.3 57.6 9.6 18.2 8.4 model | metrics
TFA w/ cos 2 41.1 65.5 44.7 50.9 79.5 56.6 11.7 23.5 9.0 model | metrics
FRCN+ft-full 3 36.5 58.5 39.9 44.8 71.4 49.5 11.6 19.6 11.2 model | metrics
TFA w/ fc 3 43.0 67.8 45.9 52.5 81.1 56.9 14.3 27.7 12.8 model | metrics
TFA w/ cos 3 42.3 67.9 44.8 51.2 80.3 55.2 15.6 30.8 13.4 model | metrics

2-shot

Name Split
AP
AP50
AP75
bAP
bAP50
bAP75
nAP
nAP50
nAP75		 download
FRCN+ft-full 1 34.8 57.1 36.3 42.2 69.4 44.3 12.5 20.3 12.2 model | metrics
TFA w/ fc 1 41.1 66.9 43.2 49.1 79.5 51.6 17.2 29.1 17.8 model | metrics
TFA w/ cos 1 42.2 68.2 45.5 49.1 78.9 52.9 21.5 36.1 23.3 model | metrics
FRCN+ft-full 2 32.5 53.7 34.2 39.4 64.8 41.7 11.7 20.6 11.7 model | metrics
TFA w/ fc 2 40.1 66.3 42.0 48.7 78.7 52.1 14.3 29.0 11.7 model | metrics
TFA w/ cos 2 39.4 65.0 42.1 47.9 77.7 52.2 14.0 26.9 12.0 model | metrics
FRCN+ft-full 3 35.4 59.1 36.8 43.5 71.8 45.7 11.1 20.8 10.0 model | metrics
TFA w/ fc 3 42.7 68.9 45.0 50.9 80.6 54.4 18.1 33.6 16.7 model | metrics
TFA w/ cos 3 42.4 68.6 44.7 50.1 79.9 53.2 19.4 34.8 19.4 model | metrics

3-shot

Name Split
AP
AP50
AP75
bAP
bAP50
bAP75
nAP
nAP50
nAP75		 download
FRCN+ft-full 1 34.5 56.8 37.2 40.2 66.1 43.2 17.6 29.0 19.2 model | metrics
TFA w/ fc 1 44.6 70.3 49.4 50.7 79.2 56.3 26.2 43.6 28.8 model | metrics
TFA w/ cos 1 45.0 70.5 50.4 50.8 79.1 56.9 27.8 44.7 30.9 model | metrics
FRCN+ft-full 2 32.5 53.6 34.4 38.1 62.0 41.1 15.7 28.6 14.2 model | metrics
TFA w/ fc 2 42.0 67.7 44.8 50.3 79.2 54.8 17.1 33.4 14.6 model | metrics
TFA w/ cos 2 42.2 67.7 45.8 50.3 78.8 55.6 17.9 34.1 16.3 model | metrics
FRCN+ft-full 3 36.5 58.7 39.0 43.2 68.7 46.4 16.3 28.7 16.8 model | metrics
TFA w/ fc 3 45.9 70.8 50.0 53.0 80.3 58.5 24.5 42.5 24.4 model | metrics
TFA w/ cos 3 45.7 71.0 50.0 52.6 80.4 58.0 25.0 42.8 26.1 model | metrics

5-shot

Name Split
AP
AP50
AP75
bAP
bAP50
bAP75
nAP
nAP50
nAP75		 download
FRCN+ft-full 1 37.4 60.1 40.3 41.3 66.7 44.4 25.5 40.1 27.7 model | metrics
TFA w/ fc 1 46.7 73.4 51.1 51.4 79.2 56.7 32.4 55.7 34.3 model | metrics
TFA w/ cos 1 47.2 73.4 52.3 51.5 79.3 57.2 34.2 55.7 37.6 model | metrics
FRCN+ft-full 2 32.6 55.9 33.5 37.6 63.7 39.1 17.5 32.4 16.5 model | metrics
TFA w/ fc 2 43.0 68.3 46.5 50.7 79.3 55.6 19.9 35.5 19.0 model | metrics
TFA w/ cos 2 43.1 68.0 47.2 50.9 78.9 56.3 19.9 35.1 19.9 model | metrics
FRCN+ft-full 3 38.1 61.8 40.4 42.9 68.3 46.2 23.4 42.2 22.9 model | metrics
TFA w/ fc 3 46.3 72.3 50.4 52.7 80.2 58.3 27.1 48.7 26.5 model | metrics
TFA w/ cos 3 46.3 72.5 49.9 52.5 80.2 56.9 27.9 49.5 28.7 model | metrics

10-shot

Name Split
AP
AP50
AP75
bAP
bAP50
bAP75
nAP
nAP50
nAP75		 download
FRCN+ft-full 1 37.5 60.9 40.0 40.4 66.0 42.9 28.7 45.5 31.2 model | metrics
TFA w/ fc 1 46.6 73.2 51.0 50.9 78.6 56.1 33.5 57.0 35.6 model | metrics
TFA w/ cos 1 47.3 72.8 52.2 51.5 78.4 56.8 34.5 56.0 38.1 model | metrics
FRCN+ft-full 2 32.2 55.5 33.3 36.0 61.0 37.7 20.8 38.8 20.4 model | metrics
TFA w/ fc 2 42.7 68.7 45.9 50.1 78.6 55.0 20.5 39.0 18.7 model | metrics
TFA w/ cos 2 43.3 68.6 47.2 50.7 78.5 56.1 21.1 39.1 20.5 model | metrics
FRCN+ft-full 3 37.3 60.8 39.4 41.3 67.0 43.4 25.2 42.1 27.6 model | metrics
TFA w/ fc 3 46.7 72.2 51.2 52.6 79.5 58.2 29.1 50.2 30.3 model | metrics
TFA w/ cos 3 47.0 72.4 51.7 52.8 79.9 58.5 29.4 49.8 31.3 model | metrics

Results Over 30 Seeds

1-shot

Name Split
AP
AP50
AP75
bAP
bAP50
bAP75
nAP
nAP50
nAP75
FRCN+ft-full 1 30.2 49.4 32.2 38.2 62.6 40.8 6 9.9 6.3
TFA w/ fc 1 39.6 63.5 43.2 48.7 77.1 53.7 12.2 22.9 11.6
TFA w/ cos 1 40.6 64.5 44.7 49.4 77.6 54.8 14.2 25.3 14.2
FRCN+ft-full 2 30.3 49.7 32.3 38.8 63.2 41.6 5 9.4 4.5
TFA w/ fc 2 36.2 59.6 38.7 45.6 73.8 49.4 8.1 16.9 6.6
TFA w/ cos 2 36.7 59.9 39.3 45.9 73.8 49.8 9.0 18.3 7.8
FRCN+ft-full 3 30.8 49.8 32.9 39.6 63.7 42.5 4.5 8.1 4.2
TFA w/ fc 3 39.0 62.3 42.1 49.5 77.8 54.0 7.8 15.7 6.5
TFA w/ cos 3 40.1 63.5 43.6 50.2 78.7 55.1 9.6 17.9 9.1

2-shot

Name Split
AP
AP50
AP75
bAP
bAP50
bAP75
nAP
nAP50
nAP75
FRCN+ft-full 1 30.5 49.4 32.6 37.3 60.7 40.1 9.9 15.6 10.3
TFA w/ fc 1 40.5 65.5 43.8 47.8 75.8 52.2 18.9 34.5 18.4
TFA w/ cos 1 42.6 67.1 47.0 49.6 77.3 55.0 21.7 36.4 22.8
FRCN+ft-full 2 30.7 49.7 32.9 38.4 61.6 41.4 7.7 13.8 7.4
TFA w/ fc 2 38.5 62.8 41.2 46.9 74.9 51.2 13.1 26.4 11.3
TFA w/ cos 2 39.0 63.0 42.1 47.3 74.9 51.9 14.1 27.5 12.7
FRCN+ft-full 3 31.3 50.2 33.5 39.1 62.4 42 8 13.9 7.9
TFA w/ fc 3 41.1 65.1 44.3 50.1 77.7 54.8 14.2 27.2 12.6
TFA w/ cos 3 41.8 65.6 45.3 50.7 78.4 55.6 15.1 27.2 14.4

3-shot

Name Split
AP
AP50
AP75
bAP
bAP50
bAP75
nAP
nAP50
nAP75
FRCN+ft-full 1 31.8 51.4 34.2 37.9 61.3 40.7 13.7 21.6 14.8
TFA w/ fc 1 41.8 67.1 45.4 48.2 76.0 53.1 22.6 40.4 22.4
TFA w/ cos 1 43.7 68.5 48.3 49.8 77.3 55.4 25.4 42.1 27.0
FRCN+ft-full 2 31.1 50.1 33.2 38.1 61 41.2 9.8 17.4 9.4
TFA w/ fc 2 39.4 64.2 42.0 47.5 75.4 51.7 15.2 30.5 13.1
TFA w/ cos 2 40.1 64.5 43.3 48.1 75.6 52.9 16.0 30.9 14.4
FRCN+ft-full 3 32.1 51.3 34.3 39.1 62.1 42.1 11.1 19 11.2
TFA w/ fc 3 40.4 65.4 43.1 47.8 75.6 52.1 18.1 34.7 16.2
TFA w/ cos 3 43.1 67.5 46.7 51.1 78.6 56.3 18.9 34.3 18.1

5-shot

Name Split
AP
AP50
AP75
bAP
bAP50
bAP75
nAP
nAP50
nAP75
FRCN+ft-full 1 32.7 52.5 35 37.6 60.6 40.3 17.9 28 19.2
TFA w/ fc 1 41.9 68.0 45.0 47.2 75.1 51.5 25.9 46.7 25.3
TFA w/ cos 1 44.8 70.1 49.4 50.1 77.4 55.6 28.9 47.9 30.6
FRCN+ft-full 2 31.5 50.8 33.6 37.9 60.4 40.8 12.4 21.9 12.1
TFA w/ fc 2 40.0 65.1 42.6 47.5 75.3 51.6 17.5 34.6 15.5
TFA w/ cos 2 40.9 65.7 44.1 48.6 76.2 53.3 17.8 34.1 16.2
FRCN+ft-full 3 32.4 51.7 34.4 38.5 61 41.3 14 23.9 13.7
TFA w/ fc 3 41.3 67.1 44.0 48.0 75.8 52.2 21.4 40.8 19.4
TFA w/ cos 3 44.1 69.1 47.8 51.3 78.5 56.4 22.8 40.8 22.1

10-shot

Name Split
AP
AP50
AP75
bAP
bAP50
bAP75
nAP
nAP50
nAP75
FRCN+ft-full 1 33.3 53.8 35.5 36.8 59.8 39.2 22.7 35.6 24.4
TFA w/ fc 1 42.8 69.5 46.0 47.3 75.4 51.6 29.3 52.0 29.0
TFA w/ cos 1 45.8 71.3 50.4 50.4 77.5 55.9 32.0 52.9 33.7
FRCN+ft-full 2 32.2 52.3 34.1 37.2 59.8 39.9 17 29.8 16.7
TFA w/ fc 2 41.3 67.0 44.0 48.3 76.1 52.7 20.2 39.7 18.0
TFA w/ cos 2 42.3 67.6 45.7 49.4 76.9 54.5 20.8 39.5 19.2
FRCN+ft-full 3 33.1 53.1 35.2 38 60.5 40.7 18.4 31 18.7
TFA w/ fc 3 42.2 68.3 44.9 48.5 76.2 52.9 23.3 44.6 21.0
TFA w/ cos 3 45.0 70.3 48.9 51.6 78.6 57.0 25.4 45.6 24.7

COCO Object Detection Baselines

Results on Seed 0

Base Models

Name
bAP
bAP50
bAP75		 download
Base Model 39.2 59.3 42.8 model | metrics

1-shot

Name
AP
AP50
AP75
bAP
bAP50
bAP75
nAP
nAP50
nAP75		 download
FRCN+ft-full 19.1 30.6 21.1 24.8 39.7 27.4 1.8 3.2 2.1 model | metrics
TFA w/ fc 25.7 41.0 27.7 33.3 52.8 36.0 2.9 5.7 2.8 model | metrics
TFA w/ cos 26.4 42.5 28.2 34.1 54.7 36.4 3.4 5.8 3.8 model | metrics

2-shot

Name
AP
AP50
AP75
bAP
bAP50
bAP75
nAP
nAP50
nAP75		 download
FRCN+ft-full 18.9 29.9 20.7 24.2 37.9 26.6 3.1 5.8 2.9 model | metrics
TFA w/ fc 26.1 41.4 28.3 33.3 52.3 36.4 4.3 8.5 4.1 model | metrics
TFA w/ cos 27.1 43.4 29.4 34.7 55.1 37.6 4.6 8.3 4.8 model | metrics

3-shot

Name
AP
AP50
AP75
bAP
bAP50
bAP75
nAP
nAP50
nAP75		 download
FRCN+ft-full 18.3 29.5 19.8 22.9 36.3 25.0 4.7 8.9 4.3 model | metrics
TFA w/ fc 26.9 42.5 29.5 33.7 52.5 37.2 6.7 12.6 6.6 model | metrics
TFA w/ cos 27.7 44.1 30.0 34.7 54.8 37.9 6.6 12.1 6.5 model | metrics

5-shot

Name
AP
AP50
AP75
bAP
bAP50
bAP75
nAP
nAP50
nAP75		 download
FRCN+ft-full 18.0 29.0 19.3 22.0 34.9 23.8 6.0 11.3 5.7 model | metrics
TFA w/ fc 27.5 43.6 30.0 33.9 52.8 37.2 8.4 16.0 8.4 model | metrics
TFA w/ cos 28.1 44.6 30.2 34.7 54.4 37.6 8.3 15.3 8.0 model | metrics

10-shot

Name
AP
AP50
AP75
bAP
bAP50
bAP75
nAP
nAP50
nAP75		 download
FRCN+ft-full 18.1 30.0 18.9 21.0 34.3 22.1 9.2 17.0 9.2 model | metrics
TFA w/ fc 27.9 44.6 30.4 33.9 53.1 37.4 10.0 19.2 9.2 model | metrics
TFA w/ cos 28.7 46.0 31.0 35.0 55.0 38.3 10.0 19.1 9.3 model | metrics

30-shot

Name
AP
AP50
AP75
bAP
bAP50
bAP75
nAP
nAP50
nAP75		 download
FRCN+ft-full 18.6 30.8 19.3 20.6 33.4 21.7 12.5 23.0 12.0 model | metrics
TFA w/ fc 29.7 46.8 32.5 35.1 54.2 38.9 13.4 24.7 13.2 model | metrics
TFA w/ cos 30.3 47.9 32.9 35.8 55.5 39.4 13.7 24.9 13.4 model | metrics

Results Over 10 Seeds

1-shot

Name
AP
AP50
AP75
bAP
bAP50
bAP75
nAP
nAP50
nAP75
FRCN+ft-full 16.2 25.8 17.6 21 33.3 23 1.7 3.3 1.6
TFA w/ fc 24 38.9 25.8 31.5 50.7 33.9 1.6 3.4 1.3
TFA w/ cos 24.4 39.8 26.1 31.9 51.8 34.3 1.9 3.8 1.7

2-shot

Name
AP
AP50
AP75
bAP
bAP50
bAP75
nAP
nAP50
nAP75
FRCN+ft-full 15.8 25 17.3 20 31.4 22.2 3.1 6.1 2.9
TFA w/ fc 24.5 39.3 26.5 31.4 49.8 34.3 3.8 7.8 3.2
TFA w/ cos 24.9 40.1 27 31.9 50.8 34.8 3.9 7.8 3.6

3-shot

Name
AP
AP50
AP75
bAP
bAP50
bAP75
nAP
nAP50
nAP75
FRCN+ft-full 15 23.9 16.4 18.8 29.5 20.7 3.7 7.1 3.5
TFA w/ fc 24.9 39.7 27.1 31.5 49.6 34.6 5 9.9 4.6
TFA w/ cos 25.3 40.4 27.6 32 50.5 35.1 5.1 9.9 4.8

5-shot

Name
AP
AP50
AP75
bAP
bAP50
bAP75
nAP
nAP50
nAP75
FRCN+ft-full 14.4 23 15.6 17.6 27.8 19.3 4.6 8.7 4.4
TFA w/ fc 25.6 40.7 28 31.8 49.8 35.2 6.9 13.4 6.3
TFA w/ cos 25.9 41.2 28.4 32.3 50.5 35.6 7 13.3 6.5

10-shot

Name
AP
AP50
AP75
bAP
bAP50
bAP75
nAP
nAP50
nAP75
FRCN+ft-full 13.4 21.8 14.5 16.1 25.7 17.5 5.5 10 5.5
TFA w/ fc 26.2 41.8 28.6 32 49.9 35.3 9.1 17.3 8.5
TFA w/ cos 26.6 42.2 29 32.4 50.6 35.7 9.1 17.1 8.8

30-shot

Name
AP
AP50
AP75
bAP
bAP50
bAP75
nAP
nAP50
nAP75
FRCN+ft-full 13.5 21.8 14.5 15.6 24.8 16.9 7.4 13.1 7.4
TFA w/ fc 28.4 44.4 31.2 33.8 51.8 37.6 12 22.2 11.8
TFA w/ cos 28.7 44.7 31.5 34.2 52.3 38 12.1 22 12

LVIS Object Detection Baselines

Base Models

With Repeat Sampling

Name Backbone
AP
AP50
AP75
APs
APm
APl
APc
APf		 download
Base Model w/ fc R-50 20.5 34.5 21.5 17.3 26.0 33.1 21.3 27.6 model | metrics
Base Model w/ cos R-50 20.2 33.5 20.9 17.4 25.7 32.8 20.4 27.9 model | metrics
Base Model w/ fc R-101 22.3 36.8 23.6 17.9 28.6 35.5 24.0 29.2 model | metrics
Base Model w/ cos R-101 21.7 35.6 22.7 17.7 18.4 34.9 22.3 29.6 model | metrics

No Repeat Sampling

Name Backbone
AP
AP50
AP75
APs
APm
APl
APc
APf		 download
Base Model w/ fc R-50 18.6 31.6 19.2 16.6 24.2 31.8 16.6 28.5 model | metrics
Base Model w/ cos R-50 18.0 30.1 18.7 16.0 23.9 31.4 15.3 28.7 model | metrics
Base Model w/ fc R-101 19.8 32.7 21.0 17.4 26.3 34.0 18.2 29.8 model | metrics
Base Model w/ cos R-101 19.8 32.6 20.6 17.6 25.6 34.2 17.9 30.0 model | metrics

Fine-tuned Models

With Repeat Sampling

Name Backbone
AP
AP50
AP75
APs
APm
APl
APr
APc
APf		 download
TFA w/ fc R-50 24.1 39.9 25.4 19.5 29.1 36.7 14.9 23.9 27.9 model | metrics
TFA w/ cos R-50 24.4 40.0 26.1 19.9 29.5 38.2 16.9 24.3 27.7 model | metrics
TFA w/ fc R-101 25.4 41.8 27.0 19.8 31.1 39.2 15.5 26.0 28.6 model | metrics
TFA w/ cos R-101 26.2 41.8 27.5 20.2 32.0 39.9 17.3 26.4 29.6 model | metrics

No Repeat Sampling

Name Backbone
AP
AP50
AP75
APs
APm
APl
APr
APc
APf		 download
TFA w/ fc R-50 22.3 37.8 22.2 18.5 28.2 36.6 14.3 21.1 27.0 model | metrics
TFA w/ cos R-50 22.7 37.2 23.9 18.8 27.7 37.1 15.4 20.5 28.4 model | metrics
TFA w/ fc R-101 23.9 39.3 25.3 19.5 29.5 38.6 16.2 22.3 28.9 model | metrics
TFA w/ cos R-101 24.3 39.3 25.8 20.1 30.2 39.5 18.1 21.8 29.8 model | metrics