Download model too slow, is there any way

[bigzhouj]

in run_lm_finetuning.py
transformers.file_utils - https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-cased-pytorch_model.bin not found in cache or force_download set to True, downloading to ....

[LysandreJik]

If your model download is too slow and fails, you can manually download it from our S3 using your browser, wget or cURL as an alternative method.

You can then point to a directory that has both the model weights (xxx-pytorch_model.bin) and the configuration file (xxx-config.json) instead of the checkpoint name as the argument for run_lm_finetuning.py.

[karajan1001]

The models on s3 are downloaded by botocore. And can be accelerated using a proxy. Detailed information can be found on .
Because It only supports http proxy now, other form of proxies like socks5 need to be converted to a http form.

[bigzhouj]

OSError: Couldn't reach server at 'https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-cased-config.json' to download pretrained model configuration file.

[karajan1001]

Can you open this https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-cased-config.json in your browser ?

[bigzhouj]

It can be opened in a browser

[bigzhouj]

in run_lm_finetuning.py,
probability_matrix.masked_fill_(torch.tensor(special_tokens_mask, dtype=torch.bool), value=0.0).
Why does dtyped equal torch.bool?
I have a difficulty here:
Expected object of scalar type Byte but got scalar type Bool for argument #2 'mask'

[TheEdoardo93]

Tipically, when you say masked *, you want to use boolean values (0 for absence and 1 for presence). In this particular case (rows n.144-151), you are sampling some tokens in the in each sequence for masked language modeling. For this reason, the probability_matrix variable is being set to boolean values. In fact, the first argument of the masked_fill() method is a boolean Torch tensor (i.e. the boolean vector). You can read more info in the PyTorch docs here.

For what concern your issue, post the code for reproducibility and version of TensorFlow, PyTorch, Transformers.

in run_lm_finetuning.py,
probability_matrix.masked_fill_(torch.tensor(special_tokens_mask, dtype=torch.bool), value=0.0).
Why does dtyped equal torch.bool?
I have a difficulty here:
Expected object of scalar type Byte but got scalar type Bool for argument #2 'mask'

[LysandreJik]

@bigzhouj this is probably due to a Pytorch version error. I believe bool was introduced in pytorch v1.2.0. What is your Pytorch version?