(multifit) ubuntu@ip-172-31-42-96:~/multifit$ python -m ulmfit eval --glob="wiki/de-100/models/sp15k/qrnn_nl4.m" --name nl4 --dataset-template='../cls/de-books-laser' --num-lm-epochs=20 --num-cls-epochs=8 --bs=18 --lr_sched=1cycle --label-smoothing-eps=0.1 Processing data/wiki/de-100/models/sp15k/qrnn_nl4.m ../cls/de-books-laser Max vocab: 15000 Cache dir: /home/ubuntu/multifit/data/cls/de-books-laser/models/sp15k Model dir: /home/ubuntu/multifit/data/cls/de-books-laser/models/sp15k/qrnn_nl4.m Training Validation set not found using 10% of trn (34000, 3) (1800, 2) bunch_path /home/ubuntu/multifit/data/cls/de-books-laser/models/sp15k/lm Running tokenization lm... Data lm, trn: 30600, val: 3400 bunch_path /home/ubuntu/multifit/data/cls/de-books-laser/models/sp15k/cls Running tokenization cls... Data cls, trn: 1800, val: 200 bunch_path /home/ubuntu/multifit/data/cls/de-books-laser/models/sp15k/tst Running tokenization tst... Data tst, trn: 200, val: 2000 Size of vocabulary: 15000 First 20 words in vocab: ['xxunk', 'xxpad', 'xxbos', 'xxfld', 'xxmaj', 'xxup', 'xxrep', 'xxwrep', '', '▁', '▁.', '▁,', '▁der', '▁die', '▁:', 'en', '▁und', 's', 'er', '▁in'] Training args: {'clip': 0.12, 'alpha': 2, 'beta': 1, 'drop_mult': 0.3} dps: {'output_p': 0.25, 'hidden_p': 0.1, 'input_p': 0.2, 'embed_p': 0.02, 'weight_p': 0.15} Loading pretrained model Bptt 70 Training lm from: [PosixPath('/home/ubuntu/multifit/data/wiki/de-100/models/sp15k/qrnn_nl4.m/lm_best'), PosixPath('/home/ubuntu/multifit/data/wiki/de-100/models/sp15k/qrnn_nl4.m/../itos')] epoch train_loss valid_loss accuracy time 0 5.027264 3.411652 0.623223 00:48 epoch train_loss valid_loss accuracy time 0 4.462070 3.315628 0.628968 01:12 1 4.205690 3.207726 0.637905 01:20 2 3.988150 3.099567 0.649690 01:23 3 3.837916 3.010923 0.661335 01:22 4 3.754120 2.924970 0.672032 01:23 5 3.599374 2.864226 0.680659 01:23 6 3.537129 2.797134 0.689113 01:23 7 3.510296 2.742251 0.697188 01:23 8 3.390882 2.691760 0.705573 01:22 9 3.324962 2.644332 0.713095 01:22 10 3.269121 2.598188 0.720428 01:22 11 3.290480 2.559261 0.727622 01:22 12 3.146980 2.524656 0.733437 01:22 13 3.159347 2.496734 0.738199 01:22 14 3.092019 2.468077 0.744410 01:22 15 3.039952 2.452549 0.748395 01:22 16 3.025997 2.437596 0.751881 01:22 17 3.002417 2.431580 0.753399 01:22 18 2.963698 2.428882 0.753882 01:22 19 2.970906 2.428425 0.753882 01:22 /home/ubuntu/multifit/data/cls/de-books-laser/models/sp15k Saving info /home/ubuntu/multifit/data/cls/de-books-laser/models/sp15k/qrnn_nl4.m/info.json Single training schedule epoch train_loss valid_loss accuracy time 0 0.584747 0.522357 0.790000 00:32 1 0.502485 0.485053 0.795000 00:33 2 0.414943 0.462057 0.830000 00:32 3 0.326147 0.519660 0.825000 00:32 4 0.277749 0.461949 0.830000 00:33 5 0.255194 0.446906 0.840000 00:33 6 0.241174 0.445065 0.845000 00:33 7 0.226501 0.445977 0.840000 00:32 Saving models at /home/ubuntu/multifit/data/cls/de-books-laser/models/sp15k/qrnn_nl4.m Loss and accuracy using (cls_best): [0.662472, tensor(0.7055), tensor(0.7299), tensor(0.6762)] [0.3479208, tensor(0.8400), tensor(0.8632), tensor(0.8072)] name tst_accuracy tst_loss val_accuracy val_loss 0 data/cls/de-books-laser/models/sp15k/qrnn_nl4.m 0.7055 0.662472 0.84 0.347921 ds de-books-las best 70.55 max 70.55 avg 70.55