-
-
Notifications
You must be signed in to change notification settings - Fork 12.7k
๐ fix: Improve stability of Cloudflare Workers AI #7450
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weโll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
@BrandonStudio is attempting to deploy a commit to the LobeChat Community Team on Vercel. A member of the Team first needs to authorize it. |
๐ @BrandonStudio Thank you for raising your pull request and contributing to our Community |
TestGru AssignmentSummary
Tip You can |
Codecov ReportAll modified and coverable lines are covered by tests โ
Additional details and impacted files@@ Coverage Diff @@
## main #7450 +/- ##
=======================================
Coverage 89.49% 89.49%
=======================================
Files 760 760
Lines 54191 54193 +2
Branches 3466 3467 +1
=======================================
+ Hits 48500 48502 +2
Misses 5691 5691
Flags with carried forward coverage won't be shown. Click here to find out more. โ View full report in Codecov by Sentry. ๐ New features to boost your workflow:
|
โค๏ธ Great PR @BrandonStudio โค๏ธ The growth of project is inseparable from user feedback and contribution, thanks for your contribution! If you are interesting with the lobehub developer community, please join our discord and then dm @arvinxx or @canisminor1990. They will invite you to our private developer channel. We are talking about the lobe-chat development or sharing ai newsletter around the world. |
### [Version 1.82.6](v1.82.5...v1.82.6) <sup>Released on **2025-04-24**</sup> #### ๐ Bug Fixes - **misc**: Improve stability of Cloudflare Workers AI. <br/> <details> <summary><kbd>Improvements and Fixes</kbd></summary> #### What's fixed * **misc**: Improve stability of Cloudflare Workers AI, closes [#7450](#7450) ([3e81e56](3e81e56)) </details> <div align="right"> [](#readme-top) </div>
๐ This PR is included in version 1.82.6 ๐ The release is available on: Your semantic-release bot ๐ฆ๐ |
๐ป ๅๆด็ฑปๅ | Change Type
๐ ๅๆด่ฏดๆ | Description of Change
Skip data chunk when
response
is missing๐ ่กฅๅ ไฟกๆฏ | Additional Information
This does not fix error in Cloudflare Workers AI. The primary reason of truncation is
max_tokens
limit, which defaults to256
.I'm not sure whether it is good to arbitratrily set a larger limit, say
2048
, because it may still be short, and it may overflow to the context window.I believe the best solution is to give an option in frontend, enabling users to set the
max_tokens
by themselves. @arvinxxThere's another issue with reasoning models in Cloudflare: missing of opening
<think>
tag. This is an expected behaviour by design, see https://discord.com/channels/595317990191398933/1105477009964027914/1354837625554604143. However, this behaviour may change, so I'm not going to prefix<think>
for now