Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

执行python init_database.py --recreate-vs 命令报错,ERROR: init() got an unexpected keyword argument '_name_or_path' AttributeError: 'NoneType' object has no attribute 'conjugate' #3208

Closed
zcs007 opened this issue Mar 6, 2024 · 3 comments
Assignees
Labels
bug Something isn't working

Comments

@zcs007
Copy link

zcs007 commented Mar 6, 2024

2024-03-06 17:28:41,134 - SentenceTransformer.py[line:66] - INFO: Load pretrained SentenceTransformer: embeddings
2024-03-06 17:28:41,469 - embeddings_api.py[line:40] - ERROR: init() got an unexpected keyword argument '_name_or_path'
AttributeError: 'NoneType' object has no attribute 'conjugate'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/hduser/langchainchatchat/init_database.py", line 107, in
folder2db(kb_names=args.kb_name, mode="recreate_vs", embed_model=args.embed_model)
File "/home/hduser/langchainchatchat/server/knowledge_base/migrate.py", line 128, in folder2db
files2vs(kb_name, kb_files)
File "/home/hduser/langchainchatchat/server/knowledge_base/migrate.py", line 113, in files2vs
kb.add_doc(kb_file=kb_file, not_refresh_vs_cache=True)
File "/home/hduser/langchainchatchat/server/knowledge_base/kb_service/base.py", line 130, in add_doc
doc_infos = self.do_add_doc(docs, **kwargs)
File "/home/hduser/langchainchatchat/server/knowledge_base/kb_service/milvus_kb_service.py", line 81, in do_add_doc
ids = self.milvus.add_documents(docs)
File "/home/hduser/miniconda3/envs/chatglm3/lib/python3.9/site-packages/langchain_core/vectorstores.py", line 119, in add_documents
return self.add_texts(texts, metadatas, **kwargs)
File "/home/hduser/miniconda3/envs/chatglm3/lib/python3.9/site-packages/langchain_community/vectorstores/milvus.py", line 531, in add_texts
embeddings = self.embedding_func.embed_documents(texts)
File "/home/hduser/langchainchatchat/server/knowledge_base/kb_service/base.py", line 420, in embed_documents
return normalize(embeddings).tolist()
File "/home/hduser/langchainchatchat/server/knowledge_base/kb_service/base.py", line 37, in normalize
norm = np.linalg.norm(embeddings, axis=1)
File "<array_function internals>", line 200, in norm
File "/home/hduser/miniconda3/envs/chatglm3/lib/python3.9/site-packages/numpy/linalg/linalg.py", line 2541, in norm
s = (x.conj() * x).real
TypeError: loop of ufunc does not support argument 0 of type NoneType which has no callable conjugate method

执行python init_database.py --recreate-vs 命令报错

复现问题的步骤 / Steps to Reproduce
python init_database.py --recreate-vs

环境信息 / Environment Information

  • langchain-ChatGLM 版本/commit 号:0.2.10
  • 是否使用 Docker 部署(是/否):否
  • 使用的模型(ChatGLM2-6B / Qwen-7B 等):ChatGLM3-6B
  • 使用的 Embedding 模型(moka-ai/m3e-base 等):moka-ai/m3e-base / Embedding model used (moka-ai/m3e-base, etc.): moka-ai/m3e-base
  • 使用的向量库类型 (faiss / milvus / pg_vector 等): milvus
  • 操作系统及版本 / Operating system and version: Linux algo 5.4.0-171-generic 一点建议 #189-Ubuntu SMP Fri Jan 5 14:23:02 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
  • Python 版本 / Python version:3.9.18
  • 其他相关环境信息 / Other relevant environment information:

附加信息 / Additional Information
添加与问题相关的任何其他信息 / Add any other information related to the issue.

@zcs007 zcs007 added the bug Something isn't working label Mar 6, 2024
Copy link

dosubot bot commented Mar 6, 2024

Based on the error message you provided, it seems like there are two main issues: init() got an unexpected keyword argument '_name_or_path' and AttributeError: 'NoneType' object has no attribute 'conjugate'.

The first error suggests that there's an issue with how the embedding model is being initialized. The unexpected keyword argument _name_or_path implies that the function or class constructor being called does not accept this argument, or it's being passed incorrectly. Ensure that the version of the embedding model or library you are using in EMBEDDING_MODEL (specified in configs/model_config.py) supports the arguments being passed during its initialization.

The second error, AttributeError: 'NoneType' object has no attribute 'conjugate', suggests that an operation is being attempted on a None object instead of a numerical array or a similar object that supports the conjugate method. This could happen if the embeddings are not correctly loaded or generated, resulting in None being passed to the normalize function.

The normalize function in base.py has the following signature:

def normalize(embeddings: List[List[float]]) -> np.ndarray:

It expects a list of lists of floats as input, where each inner list represents a vector of embeddings. If None is passed to this function, it will result in the AttributeError you're seeing.

To resolve these issues, you can follow these steps:

  1. Check the Embedding Model Initialization: Verify the initialization of the embedding model and ensure it supports the arguments being passed.

  2. Review the normalize Function: Verify the implementation of the normalize function in base.py to ensure it correctly handles cases where embeddings might not be loaded or generated as expected.

  3. Verify Embedding Model Compatibility: Ensure that the embedding model specified by EMBEDDING_MODEL is compatible with the version of the library or framework you are using.

  4. Update Dependencies: Make sure all relevant Python packages are updated to their latest versions that support the functionality you are using.

  5. Debug Initialization Parameters: Temporarily modify the code where the embedding model is initialized to print or log the parameters being passed to the model's constructor.

  6. Consult Documentation: Review the documentation for the embedding model and the library or framework it's part of.

If you continue to encounter issues, please provide more details about the steps leading up to the error, and any additional error messages or logs that might help diagnose the problem.

For more information, you can refer to the following issues in the Langchain-Chatchat repository that discuss similar problems:

I hope this helps! If you have any other questions, feel free to ask.

Sources

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

@zcs007 zcs007 changed the title [BUG] 简洁阐述问题 / Concise description of the issue 执行python init_database.py --recreate-vs 命令报错,ERROR: init() got an unexpected keyword argument '_name_or_path' AttributeError: 'NoneType' object has no attribute 'conjugate' Mar 6, 2024
@zcs007
Copy link
Author

zcs007 commented Mar 12, 2024

@dosu-bot

@chunshengding
Copy link

同样的问题

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants