Clean install, nothing boots when i do 'python generate.py'

[Domsmasher1]

When i try to run the start script i get this error message

(h2ogpt) C:\Users\domin\Documents\aiGen\h2ogpt>python generate.py
Fontconfig error: Cannot load default config file: No such file: (null)
C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\pydub\utils.py:170: RuntimeWarning: Couldn't find ffmpeg or avconv - defaulting to ffmpeg, but may not work
  warn("Couldn't find ffmpeg or avconv - defaulting to ffmpeg, but may not work", RuntimeWarning)
WARNING:langchain_community.utils.user_agent:USER_AGENT environment variable not set, consider setting it to identify your requests.
Windows fatal exception: code 0xc0000139

Current thread 0x000007e4 (most recent call first):
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1176 in create_module
  File "<frozen importlib._bootstrap>", line 571 in module_from_spec
  File "<frozen importlib._bootstrap>", line 674 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\awq\modules\linear.py", line 4 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\awq\models\base.py", line 16 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\awq\models\mpt.py", line 1 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\awq\models\__init__.py", line 1 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 992 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\awq\__init__.py", line 2 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 992 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 992 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\peft\tuners\lora\awq.py", line 26 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\peft\tuners\lora\model.py", line 50 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\peft\tuners\lora\__init__.py", line 20 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\peft\tuners\__init__.py", line 21 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 992 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 992 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\peft\mapping.py", line 22 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\peft\auto.py", line 32 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\peft\__init__.py", line 22 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\transformers\trainer.py", line 215 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "<frozen importlib._bootstrap>", line 1050 in _gcd_import
  ...
Traceback (most recent call last):
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\transformers\utils\import_utils.py", line 1603, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\transformers\trainer.py", line 215, in <module>
    from peft import PeftModel
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\peft\__init__.py", line 22, in <module>
    from .auto import (
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\peft\auto.py", line 32, in <module>
    from .mapping import MODEL_TYPE_TO_PEFT_MODEL_MAPPING
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\peft\mapping.py", line 22, in <module>
    from peft.tuners.xlora.model import XLoraModel
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\peft\tuners\__init__.py", line 21, in <module>
    from .lora import LoraConfig, LoraModel, LoftQConfig, LoraRuntimeConfig
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\peft\tuners\lora\__init__.py", line 20, in <module>
    from .model import LoraModel
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\peft\tuners\lora\model.py", line 50, in <module>
    from .awq import dispatch_awq
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\peft\tuners\lora\awq.py", line 26, in <module>
    from awq.modules.linear import WQLinear_GEMM
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\awq\__init__.py", line 2, in <module>
    from awq.models.auto import AutoAWQForCausalLM
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\awq\models\__init__.py", line 1, in <module>
    from .mpt import MptAWQForCausalLM
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\awq\models\mpt.py", line 1, in <module>
    from .base import BaseAWQForCausalLM
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\awq\models\base.py", line 16, in <module>
    from awq.modules.linear import WQLinear_GEMM, WQLinear_GEMV
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\awq\modules\linear.py", line 4, in <module>
    import awq_inference_engine  # with CUDA kernels
ImportError: DLL load failed while importing awq_inference_engine: The specified procedure could not be found.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\domin\Documents\aiGen\h2ogpt\generate.py", line 20, in <module>
    entrypoint_main()
  File "C:\Users\domin\Documents\aiGen\h2ogpt\generate.py", line 16, in entrypoint_main
    H2O_Fire(main)
  File "C:\Users\domin\Documents\aiGen\h2ogpt\src\utils.py", line 79, in H2O_Fire
    fire.Fire(component=component, command=args)
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\fire\core.py", line 143, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\fire\core.py", line 477, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\fire\core.py", line 693, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
  File "C:\Users\domin\Documents\aiGen\h2ogpt\src\gen.py", line 2055, in main
    model=get_embedding(use_openai_embedding, hf_embedding_model=hf_embedding_model,
  File "C:\Users\domin\Documents\aiGen\h2ogpt\src\gpt_langchain.py", line 550, in get_embedding
    embedding = HuggingFaceBgeEmbeddings(model_name=hf_embedding_model,
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\langchain_community\embeddings\huggingface.py", line 287, in __init__
    import sentence_transformers
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\sentence_transformers\__init__.py", line 17, in <module>
    from sentence_transformers.trainer import SentenceTransformerTrainer
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\sentence_transformers\trainer.py", line 13, in <module>
    from transformers import EvalPrediction, PreTrainedTokenizerBase, Trainer, TrainerCallback
  File "<frozen importlib._bootstrap>", line 1075, in _handle_fromlist
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\transformers\utils\import_utils.py", line 1593, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "C:\Users\domin\miniconda3\envs\h2ogpt\lib\site-packages\transformers\utils\import_utils.py", line 1605, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.trainer because of the following error (look up to see its traceback):
DLL load failed while importing awq_inference_engine: The specified procedure could not be found.

I have gone though and followed eveything here https://github.com/h2oai/h2ogpt/blob/main/docs/README_WINDOWS.md, and eveything else ran without issue.

Any ideas?

[pseudotensor]

Same issues as here:

#1835
#1561

Maybe try loading model from UI, the other person said.

If anyone with windows can help figure it out would be great.

[Domsmasher1]

The issue is i dont even get to a stage where the ui boots, it crashes before that stage, unlike the other two which seem to get to the ui stage

[pseudotensor]

Try what was done for ooba:

oobabooga/text-generation-webui#4253 (comment)

pip uninstall autoawq
git clone https://github.com/casper-hansen/AutoAWQ
cd AutoAWQ
pip install -e .