Sparse-Vision Transformer

Pytorch implementation of Vision Transformer with Sparse Regularization. Pretrained pytorch weights are provided which are converted from original jax/flax weights. Pretrained weight can be downloaded in Vision Transformer - Pytorch

Cite

If you find this helpful, please cite this paper:

@misc{prasetyo2023sparse,
      title={Sparse then Prune: Toward Efficient Vision Transformers}, 
      author={Yogi Prasetyo and Novanto Yudistira and Agus Wahyu Widodo},
      year={2023},
      eprint={2307.11988},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}