Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: Message role always = assistant #5621

Closed
yiyangnlp opened this issue Sep 10, 2024 · 5 comments · Fixed by #5618
Closed

[Bug]: Message role always = assistant #5621

yiyangnlp opened this issue Sep 10, 2024 · 5 comments · Fixed by #5618
Labels
bug Something isn't working

Comments

@yiyangnlp
Copy link

yiyangnlp commented Sep 10, 2024 •
edited
Loading

What happened?

This line of code:

litellm/litellm/types/utils.py

Line 328 in b1ecfe0

"role": "assistant",

Should be "role": role

Relevant log output

No response

Twitter / LinkedIn details

No response
@yiyangnlp yiyangnlp added the bug Something isn't working label Sep 10, 2024
@krrishdholakia
Copy link
Contributor

This is not a bug. The role of the llm response message is always assistant.

If you see a counterexample, please share it @yiyangnlp

@yiyangnlp
Copy link
Author

yiyangnlp commented Sep 10, 2024 •
edited
Loading

If you do:

msg = Message(role="user", content="Hi! How can I help you today?")
print (msg)

>>> Message(role="assistant", content="Hi! How can I help you today?")

The role of msg is always "assistant"

Is this expected?

@yiyangnlp
Copy link
Author

BTW, this makes Bedrock not work with Message. I had to use Dict instead of Message

@krrishdholakia
Copy link
Contributor

Oh! i see what you mean - you're trying to use the Message object yourself

krrishdholakia added a commit that referenced this issue Sep 10, 2024
@krrishdholakia
fix(types/utils.py): allow passing role for message object
fcbed92 
#5621
@krrishdholakia krrishdholakia mentioned this issue Sep 11, 2024
Merged
krrishdholakia added a commit that referenced this issue Sep 11, 2024
@krrishdholakia
LiteLLM Minor Fixes and Improvements (09/10/2024) (#5618)
0295a22 
* fix(cost_calculator.py): move to debug for noisy warning message on cost calculation error

Fixes #5610

* fix(databricks/cost_calculator.py): Handles model name issues for databricks models

* fix(main.py): fix stream chunk builder for multiple tool calls

Fixes #5591

* fix: correctly set user_alias when passed in

Fixes #5612

* fix(types/utils.py): allow passing role for message object

#5621

* fix(litellm_logging.py): Fix langfuse logging across multiple projects

Fixes issue where langfuse logger was re-using the old logging object

* feat(proxy/_types.py): support adding key-based tags for tag-based routing

Enable tag based routing at key-level

* fix(proxy/_types.py): fix inheritance

* test(test_key_generate_prisma.py): fix test

* test: fix test

* fix(litellm_logging.py): return used callback object
@yiyangnlp
Copy link
Author

I now ran into this error that is related:

pydantic_core._pydantic_core.ValidationError: 1 validation error for Message
role
  Input should be 'assistant' [type=literal_error, input_value='system', input_type=str]
    For further information visit https://errors.pydantic.dev/2.8/v/literal_error

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

LiteLLM Minor Fixes and Improvements (09/10/2024) BerriAI/litellm
2 participants
@krrishdholakia @yiyangnlp