AssertionError on filetype=anki + Local LLM Integration Issues

[SiHaag]

Hello dear developer, I am unfortunately encountering issues with wdoc Querying Anki Database Using Local LLM.

System Information
• OS: macOS (M1 MacBook Air, 8GB RAM)
• Python Version: 3.11
• Wdoc Version: 2.5.7
• LLM Backend: Ollama (running deepseek-coder:1.3b as API on http://localhost:4000)
• Anki Profile Name: Neumed
• Command Used:
wdoc --task=query --filetype=anki --anki_profile "Neumed" --llms_api_bases='{"model": "http://localhost:4000"}'

Issue Description

I am trying to query my Anki collection (Neumed - name of one of my anki profiles) using wdoc while leveraging Ollama’s DeepSeek model as the LLM backend. However, I encountered an AssertionError stating:

• AssertionError: Could not infer filetype of --filetype=anki. Use the 'filetype' argument.

Additionally, when running without --llms_api_bases, I get:

• Exception: No environment variable named OPENAI_API_KEY found

indicating that wdoc defaults to OpenAI’s API even when a custom API (Ollama) is specified.

Steps to Reproduce
1. Installed and configured:
• Ollama via Homebrew (brew install ollama)
• Wdoc (v2.5.7)
• DeepSeek Coder (1.3B) as my local LLM backend
2. Verified that the model is running via:

• curl -X GET http://localhost:4000/v1/models
  Response:
• {"data":[{"id":"ollama/deepseek-coder:1.3b","object":"model","created":1677610602,"owned_by":"openai"}],"object":"list"}
  3. Tried querying Anki with:
• wdoc --task=query --filetype=anki --anki_profile "Neumed" --llms_api_bases='{"model": "http://localhost:4000"}'
  4. Result: AssertionError for filetype=anki.

Expected Behavior
• The command should correctly load and search the Anki collection for relevant cards.
• The custom LLM (DeepSeek via Ollama) should be used without requiring OPENAI_API_KEY.
• The filetype=anki argument should be inferred correctly.

Observed Issues
1. filetype=anki is not recognized despite being a documented option.
2. Custom LLM backend (Ollama) is not seamlessly integrated; wdoc still expects an OpenAI API key.
3. There are no clear examples in the docs for querying Anki decks. The documentation could use an explicit example of querying Anki with a local LLM instead of OpenAI.

Suggested Fixes
• Explicitly document how to query Anki with a local LLM backend.
• Improve filetype inference for anki or allow manual specification.
• Ensure that llms_api_bases fully overrides OpenAI as the default LLM backend.

Additional Debugging Information
• When running:

• wdoc --task=list --filetype=anki
  Error: AssertionError: invalid task value

  • I also tested with:

• wdoc --task=query --filetype=anki --anki_profile "Neumed"
  Error: Exception: No environment variable named OPENAI_API_KEY found

What I Need Help With
1. Is filetype=anki fully implemented in wdoc v2.5.7, or does it require a workaround?
2. How do I properly specify an Ollama backend for wdoc queries?
3. Could you provide an example of a working Anki query using a local LLM?

Thank you once again @thiswillbeyourgithub! ❤ I hope you find the time to help with this issue.

Sihaag

[thiswillbeyourgithub]

Hi,

1. Can you try with wdoc --task=query --filetype anki --anki_profile "Neumed" --llms_api_bases='{"model": "http://localhost:4000"} (so the exact same command but with a space after filetype)
2. Please always provide the full error trace, not just the error message
3. You are oveoading model to use ollama but that's not all the model that are used. There is also embeddings, query eval, etc that default to openai's

[SiHaag]

Thanks for the prompt replies!

1. I am attaching the text output error that occurred after I ran this command. - PFA - Terminal output 03.
2. Please let me know if you can access those from this link(I didn't think it through to save the errors as they were happening coz I was trying to resolve them with chatgpt, I will mind that going forward)- https://chatgpt.com/canvas/shared/67b3ab3f80488191a0f3d7895803b752
3. Chatgpt suggested me to run this command after reading your response -
   wdoc --task=query --filetype anki --anki_profile "Neumed"
   --llms_api_bases='{"model": "http://localhost:4000", "query_eval_model": "http://localhost:4000", "embed_model": "http://localhost:4000"}'
   So I am also attaching the output error of that titled - "Terminal - overriding other models"
   I will be honest I didn't understand at all what you meant by that so I just followed what chatgpt suggested me based on your response.

Terminal - overriding other models.txt

Terminal output 03.txt

Please, another thank you for your infectious generosity! 🙇

Sihaag

[thiswillbeyourgithub]

I just pushed version 2.6.0 that should fix that. I ended up squashing many more bugs and simplifying the ollama workflow. Thanks a lot.

I'm closing this but feel free to re open or create a new one if needed.

[SiHaag]

Hey I'm sorry there are still issues🥲.
I asked chatgpt to go through the updated version docs and based on it, it suggested this command to me, and I assume the difference was in the ports(4000 earlier vs 11434 for now, right?) -

• wdoc --task=query --filetype anki --anki_profile "YourProfileName"
  --llms_api_bases='{"model": "http://localhost:11434", "query_eval_model": "http://localhost:11434", "embeddings": "http://localhost:11434"}'
  The test run with this command is attached in - "7with ollama.txt"

According to chatgpt-

this is the issue according to it.

And it suggested this solution -

But I thought I better seek your assistance than make these changes with little knowledge.

The issue summarised by it is as follows-
I'm using WDoc with Ollama (DeepSeek-Coder 1.3B) as my backend and set --1lms_api_bases='{"model": "http://
localhost:11434"}', which works for generating responses. However, WDoc is still attempting to use OpenAl embeddings
(text-embedding-3-small), leading to authentication errors since I don't have an OpenAl API key.

I would like to fully switch WDoc to Ollama, including embeddings and query evaluation. Could you clarify:

1. How to force WDoc to use FAISS embeddings instead of OpenAl? (--embed_model "faiss")

2. If FAISS isn't supported, what's the best way to disable embeddings entirely? (--disable_embeddings?)

3. Does WDoc support local embedding models under Ollama, or is OpenAl the only option?

4. Should I switch to a different Ollama model (deepseek-coder:1.3b) for better compatibility?

5. Are there any additional parameters needed to stop WDoc from calling OpenAl for embeddings?

I am attaching 3 other .txt outputs from previous interactions but made a judgement call on asking you to view "7with ollama" first as the initial attempts were made using litellm and i later remembered you specifically mentioned making it more compatible with ollama so only the last txt output was with ollama.

7with ollama.txt

output 6 with recommended command but with litellm.txt

output 4.txt

And as usual, my today's vote of heartfelt gratitude! 🙏🏼

SiHaag

[thiswillbeyourgithub]

1 are you sure you updated?
2 you don't specify the ollama embedding model. Forget about the llm_api_bases you shoumdn't need it here. Take inspiration from the latest readme on how to use ollama