New install of WSL error with missing Tokeniser

[manassm]

Describe the bug

I was using the previous version of the webui with one-click install method until I updated it and it stopped working.
Hence, I tried the new WSL method to install it on my windows system.
But when I thought everything was complete and I tried to run it, it says that some Tokenisers are missing, I guess.
(I don't have CS background so I don't really know the details of what I am talking about.)
Please help to resolve this issue.

Is there an existing issue for this?

• I have searched the existing issues

Reproduction

(base) ssm05111@username:$ conda activate textgen
(textgen) ssm05111@username:$ cd text-generation-webui/
(textgen) ssm05111@username:~/text-generation-webui$ python server.py

Screenshot

Logs

Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
================================================================================
/home/ssm05111/miniconda3/envs/textgen/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: /home/ssm05111/miniconda3/envs/textgen did not contain libcudart.so as expected! Searching further paths...
  warn(msg)
/home/ssm05111/miniconda3/envs/textgen/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('unix')}
  warn(msg)
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching /usr/local/cuda/lib64...
/home/ssm05111/miniconda3/envs/textgen/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/usr/local/cuda/lib64')}
  warn(msg)
CUDA SETUP: WARNING! libcuda.so not found! Do you have a CUDA driver installed? If you are on a cluster, make sure you are on a CUDA machine!
/home/ssm05111/miniconda3/envs/textgen/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: No libcudart.so found! Install CUDA or the cudatoolkit package (anaconda)!
  warn(msg)
/home/ssm05111/miniconda3/envs/textgen/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: No GPU detected! Check your CUDA paths. Proceeding to load CPU-only library...
  warn(msg)
CUDA SETUP: Loading binary /home/ssm05111/miniconda3/envs/textgen/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cpu.so...
/home/ssm05111/miniconda3/envs/textgen/lib/python3.10/site-packages/bitsandbytes/cextension.py:31: UserWarning: The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers and GPU quantization are unavailable.
  warn("The installed version of bitsandbytes was compiled without GPU support. "
Loading llama-7b-hf...
Loading checkpoint shards: 100%|████████████████████████████████████████████████████████| 33/33 [00:08<00:00,  3.96it/s]
Traceback (most recent call last):
  File "/home/ssm05111/text-generation-webui/server.py", line 236, in <module>
    shared.model, shared.tokenizer = load_model(shared.model_name)
  File "/home/ssm05111/text-generation-webui/modules/models.py", line 163, in load_model
    tokenizer = AutoTokenizer.from_pretrained(Path(f"models/{shared.model_name}/"))
  File "/home/ssm05111/miniconda3/envs/textgen/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 677, in from_pretrained
    raise ValueError(
ValueError: Tokenizer class LLaMATokenizer does not exist or is not currently imported.

System Info

NVidia 4080
Windows 11

[CrazyKrow]

I had the same problem, you have to rename "LLaMATokenizer" to "LlamaTokenizer" on the tokenizer_config.json from your model. And if you still have CUDA problems try conda install cudatoolkit

[CrazyKrow]

just type explorer.exe on the wsl terminal and then you should be able to view the files inside the file explorer window. Then you can edit the tokenizer_config.json with notepad.
https://www.fourco.nl/blogs/access-to-wsl-filesystem-in-explorer/

[manassm]

Thanks for the help. I edited the json file and installed cudatoolkit. It seems like it is kinda working, but I still get these warning signs that I'm not sure if I need to worry about or not.

[cpwilson1989]

Not sure if you've seen this already, but this helped me with that error:
156#issuecomment-1462329713

[manassm]

@cpwilson1989 I did see that, thanks for the notice though.
I looked for file named libbitsandbytes_cpu.so located in linux machine \wsl.localhost\Ubuntu\home\username\miniconda3\envs\textgen\lib\python3.10\site-packages\bitsandbytes, deleted it, and changed the file name of libbitsandbytes_cuda117.so into libbitsandbytes_cpu.so. It now seems to work (there is still few lines of error but the UI seems to work). Thank you.