README.md

DistilBART

http://arxiv.org/abs/2010.13002

More info can be found here.

Speedup DistilBART (Huggingface Transformers version) by using FastSeq

• Speed on single NVIDIA-V100-16GB

  BatchSize	64	128
  transformers-4.12.0	5.5 samples/s	OOM
  above + fastseq	17.8 samples/s	19.1 samples/s

Model

sshleifer/distilbart-cnn-12-6 from model hub.

Task

CNN/DM validation data

Setting

$ fastseq-generate-for-transformers \
    sshleifer/distilbart-cnn-12-6 \
    cnn_dm/val.source \
    out.summary \
    --reference_path cnn_dm/val.target \
    --device cuda \
    --bs BATCH_SIZE \
    --fp16 \
    --score_path out.score \
    --task summarization

Baseline speed number is obtained by running Transformers v4.12.0 code.