This repository contains PyTorch evaluation code, training code and pretrained models.
We will further upload results on other model architectures.
cd ./deit
Before using it, make sure you have the pytorch-image-models [timm] package by Ross Wightman installed. Note that our work relies of the augmentations proposed in this library.
We provide models pretrained on ImageNet 2012 and finetune the checkpoint on segmentation datasets.
| name | acc@1 | #params | url |
|---|---|---|---|
| SWIN-Large | 87.4 | 197M | model |
We finetune the checkpoint and get following results.
| name | mIoU | mIoU (ms + flip) | #params | url |
|---|---|---|---|---|
| ADE20K | 83.0 | 54.4 | 234M | model, log |
| CityScapes | 82.9 | 83.9 | 234M | model, log |
For more details, please refer to README.
We provide models pretrained on ImageNet 2012. More models will be uploaded.
| name | acc@1 | acc@5 | #params | url |
|---|---|---|---|---|
| VIT-B12 | 82.9 | 96.3 | 86M | model |
| VIT-B24 | 83.3 | 96.4 | 172M | model |
| VIT-B12-384 | 84.2 | 97.0 | 86M | model |
We finetune the checkpoint proposed by VIT.
| name | acc@1 | acc@5 | #params | url |
|---|---|---|---|---|
| VIT-L24 | 83.9 | 96.7 | 305M | model |
| VIT-L24-384 | 85.4 | 96.7 | 305M | model |
For Deit-B12, run:
python -m torch.distributed.launch --nproc_per_node=XX --master_port=XX --use_env main.py --model deit_base_patch16_224 --aa rand-m9-mstd0.5-inc1 --input-size 224 --batch-size 16 --num_workers 2 --data-path path --output_dir output_dir --resume model.pth --eval
giving
Acc@1 82.928 Acc@5 96.342 loss 0.721
The training code is not fully available and the results are currently not reproducable. Please wait for our updates.
For Deit-B12, run:
python -m torch.distributed.launch --nproc_per_node=XX --master_port=XX --use_env main.py --model deit_base_patch16_224 --aa rand-m9-mstd0.5-inc1 --input-size 224 --batch-size 72 --num_workers 4 --data-path path --output_dir output_dir -no-repeated-aug --epochs 300 --model-ema-decay 0.99996 --drop-path 0.5 --drop .0 --mixup .0 --mixup-switch-prob 0.0
and further refine the model by
python -m torch.distributed.launch --nproc_per_node=XX --master_port=XX --use_env main.py --model deit_base_patch16_224 --aa rand-m9-mstd0.5-inc1 --input-size 224 --batch-size 72 --num_workers 4 --data-path path --output_dir output_dir -no-repeated-aug --start_epoch 300 --epochs 400 --resume model.pth --model-ema-decay 0.99996 --drop-path 0.75 --drop .0 --mixup .0 --mixup-switch-prob 0.0
python -m torch.distributed.launch --nproc_per_node=XX --master_port=XX --use_env main.py --model deit_large_patch16_224 --aa rand-n1-m1-mstd0.5-inc1 --input-size 224 --batch-size 16 --num_workers 1 --data-path path --output_dir output_dir -no-repeated-aug --smoothing 1e-6 --weight-decay 1e-8 --lr 5e-5 --start_epoch 0 --reprob 1e-6 --resume vit_checkpoint --epochs 40 --model-ema-decay 0.99996 --drop-path 0. --drop .0 --mixup .0 --mixup-switch-prob 0.0 --no-use-talk
evaluate
python -m torch.distributed.launch --nproc_per_node=XX --master_port=XX --use_env main.py --model deit_large_patch16_224 --aa rand-n1-m1-mstd0.5-inc1 --input-size 224 --batch-size 16 --num_workers 1 --data-path path --output_dir output_dir -no-repeated-aug --smoothing 1e-6 --weight-decay 1e-8 --lr 5e-5 --start_epoch 0 --reprob 1e-6 --resume vit_checkpoint --epochs 40 --model-ema-decay 0.99996 --drop-path 0. --drop .0 --mixup .0 --mixup-switch-prob 0.0 --no-use-talk --eval