-
Notifications
You must be signed in to change notification settings - Fork 8.1k
Fix/anthropic colud not end with trailing whitespace at assistant content #6168
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix/anthropic colud not end with trailing whitespace at assistant content #6168
Conversation
ekzhu
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not use the message transformation?
|
Thanks for the question! (I wrote it quickly at first, but just updated the comment now.)
The reason I didn't use the transformer is because this issue specifically applies to the last AssistantMessage, and the current message transformation logic works on a per-message basis — it doesn't have visibility into the full message sequence. If we apply ["show under message\n", "blah, blah"] → ["show under message", "blah, blah"]That’s why I applied the fix after the message list is fully constructed — to avoid over-sanitizing content that’s meant to preserve formatting. Let me know if you’d prefer a different way to organize this logic! |
|
(Aside: Why did we originally design the transformer to work per-message?) That was intentional — based on the Single Responsibility Principle. Message transformers are meant to map the structure of each In contrast, message stream-level rules — like "strip trailing space from the last AssistantMessage" — depend on list-level context and vendor quirks. So they should live in a separate layer like This keeps responsibilities clear:
|
Understood, let's starts with the design in this PR and wait for usages and feedbacks. |
…itespace_at_assistant_content
|
Sounds good - thanks! |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #6168 +/- ##
==========================================
+ Coverage 77.26% 77.31% +0.04%
==========================================
Files 197 197
Lines 13783 13797 +14
==========================================
+ Hits 10650 10667 +17
+ Misses 3133 3130 -3
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Please refer to #6123 for full context. That issue outlines several design and behavioral problems with `SocietyOfMindAgent`. This DRAFT PR focuses on resolving the most critical and broken behaviors first. Here is the error list 🔍 SocietyOfMindAgent: Design Issues and Historical Comparison (v0.2 vs v0.4+) ### ✅ P1–P4 Regression Issue Table (Updated with Fixes in PR #6142) | ID | Description | Current v0.4+ Issue | Resolution in PR #6142 | Was it a problem in v0.2? | Notes | |-----|-------------|----------------------|--------------------------|----------------------------|-------| | **P1** | `inner_messages` leaks into outer team termination evaluation | `Response.inner_messages` is appended to the outer team's `_message_thread`, affecting termination conditions. Violates encapsulation. | ✅ `inner_messages` is excluded from `_message_thread`, avoiding contamination of outer termination logic. | ❌ No | Structural boundary is now enforced | | **P2** | Inner team does not execute when outer message history is empty | In chained executions, if no new outer message exists, no task is created and the inner team is skipped entirely | ✅ Detects absence of new outer message and reuses the previous task, passing it via a handoff message. This ensures the inner team always receives a valid task to execute | ❌ No | The issue was silent task omission, not summary failure. Summary succeeds as a downstream effect | | **P3** | Summary LLM prompt is built from external input only | Prompt is constructed using external message history, ignoring internal reasoning | ✅ Prompt construction now uses `final_response.inner_messages`, restoring internal reasoning as the source of summarization | ❌ No | Matches v0.2 internal monologue behavior | | **P4** | External input is included in summary prompt (possibly incorrectly) | Outer messages are used in the final LLM summarization prompt | ✅ Resolved via the same fix as P3; outer messages are no longer used for summary | ❌ No | Redundant with P3, now fully addressed | <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> ## Why are these changes needed? <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number resolve #6123 Blocked #6168 (Sometimes SoMA send last whitespace message) related #6187 <!-- For example: "Closes #1234" --> ## Checks - [ ] I've included any doc changes needed for <https://microsoft.github.io/autogen/>. See <https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed. --------- Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
Why are these changes needed?
This PR fixes a
400 - invalid_request_errorthat occurs when using Anthropic models and the final message is from the assistant and ends with trailing whitespace.Example error:
To unblock ongoing internal usage, this patch introduces an ad-hoc fix that strips trailing whitespace if the model is Anthropic and the last message is from the assistant.
Related issue number
Ad-hoc fix for issue discussed here:
#6167
Follow-up structural proposal here:
#6167
#6167 (comment)
Checks