Can't select modelname in lm studio

[edvinsyk]

Hello,

I am having trouble running specific models from lm studio.
Not sure if I am doing anything wrong if its bugged.

This command works, it picks whatever model I have loaded in lm studio. But it incorrectly says that it is using this when I run it with verbose LiteLLM completion() model= gpt-4o; provider = openai

uvx wdoc --llms_api_bases='{"model": "http://127.0.0.1:1234/v1"}' --path="~/to/paper" --filetype="auto" --task=summary

This gives me the following error:
AttributeError: module 'litellm' has no attribute 'model_by_providers'

uvx wdoc --llms_api_bases='{"model": "http://127.0.0.1:1234/v1"}' --modelname="lm_studio/deepseek-r1-distill-qwen-1.5b" --path="~/to/paper" --filetype="auto" --task=summary

I have tried different providers, but according to https://docs.litellm.ai/docs/providers/lm_studio it should be correct.

Got any ideas?

[thiswillbeyourgithub]

Hi,

Thanks to you I found a typo, I was mistakenly calling model_by_providers instead of models_by_providers. I just pushed it and made a new release at 2.5.6. This issue was happening when creating the error message so that's why we didn't know what was going on. Thanks.

But it incorrectly says that it is using this when I run it with verbose LiteLLM completion() model= gpt-4o; provider = openai

I don't understand your sentence. Please be more explicit and explain what you try, expect, get. Traces help.

If you get it to work please report back to explain to others then close the issue.

[edvinsyk]

Hey, thanks for looking at this!
I can see now that the above comment was unclear :)
When I run this command

uvx wdoc --llms_api_bases='{"model": "http://127.0.0.1:1234/v1"}' --path="~/to/paper" --filetype="auto" --task=summary --verbose
It tells ms that it is using openAI, gives me this message.

Token Counter - using OpenAI token counter, for model=gpt-3.5-turbo
LiteLLM: Utils - Counting tokens for OpenAI model=gpt-3.5-turbo

But it is using a loaded model from lm studio. I just can't name it.

I tried with the updated version, the problems remains, now with the error message:

AttributeError: module 'litellm' has no attribute 'models_by_providers'

I don't have time to dig deeper right now, but will look at it later!

[thiswillbeyourgithub]

The message is by litellm not wdoc and indicates that the tokenizer used is the one from openai. It's just to count tokens because not all models and apis publish their apis so I decided to easily fallback to openai's.

I need the trace of the error to fix that i think.

[thiswillbeyourgithub]

Oh crap it should be models_by_provider instead of models_by_providers. I broke my computer unfortunately so will push the fixed version as soon as I boot. Thanks!

[thiswillbeyourgithub]

Hello again. Sorry for the delay, I had a catastrophic failure. I just released 2.5.7 to address that typo. Dont hesitate to reopen if that was not enough. Thanks a lot for making me aware of this!