I'm getting "No relevant documents to query." one every prompt

[slavag]

Hi,
Using --langchain_mode='UserData' generated from CSV file by make_db.py and recently (after I pulled all the changes) I started to get "No relevant documents to query." on every prompt. Yesterday everything was working fine.
When switching to "ChatLLM" or "LLM" it's working fine.
Please advise.
Thanks

[pseudotensor]

Hi, if I run:

python generate.py --base_model=h2oai/h2ogpt-gm-oasst1-en-2048-falcon-7b-v3 --langchain_mode=UserData

and have an existing database for UserData, I don't see the same problem:

However, there is a change in behavior, so that before if it didn't find relevant documents the LLM would still be allowed to respond. But others, and I agree, suggested it's better to not let the LLM respond if have chosen a DB. If really want the LLM to say something, should switch to ChatLLM or LLM in the UI instead of choosing UserData or MyData.

So either your documents really are irrelevant, or something else is wrong. To test, you can change in gen.py:

cut_distanct=1.1 if langchain_mode in ['wiki_full'] else 1.64,  # FIXME, too arbitrary

to

cut_distanct=100

So there is no relevance cut, everything will be considered relevant.

[slavag]

@pseudotensor Thanks, indeed there was not related to DB question, but I expected to behave the same , like in the past. IMHO, the previous behaviour was more correct :)
Thanks

[pseudotensor]

I can certainly make it an option.

[slavag]

Thanks, that would be great.

[pseudotensor]

Added option --use_llm_if_no_docs=True