Skip to content

๐Ÿ› fix: Improve stability of Cloudflare Workers AI #7450

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weโ€™ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 24, 2025

Conversation

BrandonStudio
Copy link
Contributor

@BrandonStudio BrandonStudio commented Apr 17, 2025

๐Ÿ’ป ๅ˜ๆ›ด็ฑปๅž‹ | Change Type

  • โœจ feat
  • ๐Ÿ› fix
  • โ™ป๏ธ refactor
  • ๐Ÿ’„ style
  • ๐Ÿ‘ท build
  • โšก๏ธ perf
  • ๐Ÿ“ docs
  • ๐Ÿ”จ chore

๐Ÿ”€ ๅ˜ๆ›ด่ฏดๆ˜Ž | Description of Change

Skip data chunk when response is missing

๐Ÿ“ ่กฅๅ……ไฟกๆฏ | Additional Information

This does not fix error in Cloudflare Workers AI. The primary reason of truncation is max_tokens limit, which defaults to 256.
I'm not sure whether it is good to arbitratrily set a larger limit, say 2048, because it may still be short, and it may overflow to the context window.
I believe the best solution is to give an option in frontend, enabling users to set the max_tokens by themselves. @arvinxx

There's another issue with reasoning models in Cloudflare: missing of opening <think> tag. This is an expected behaviour by design, see https://discord.com/channels/595317990191398933/1105477009964027914/1354837625554604143. However, this behaviour may change, so I'm not going to prefix <think> for now

Copy link

vercel bot commented Apr 17, 2025

@BrandonStudio is attempting to deploy a commit to the LobeChat Community Team on Vercel.

A member of the Team first needs to authorize it.

@dosubot dosubot bot added the size:XS This PR changes 0-9 lines, ignoring generated files. label Apr 17, 2025
@lobehubbot
Copy link
Member

๐Ÿ‘ @BrandonStudio

Thank you for raising your pull request and contributing to our Community
Please make sure you have followed our contributing guidelines. We will review it as soon as possible.
If you encounter any problems, please feel free to connect with us.
้žๅธธๆ„Ÿ่ฐขๆ‚จๆๅ‡บๆ‹‰ๅ–่ฏทๆฑ‚ๅนถไธบๆˆ‘ไปฌ็š„็คพๅŒบๅšๅ‡บ่ดก็Œฎ๏ผŒ่ฏท็กฎไฟๆ‚จๅทฒ็ป้ตๅพชไบ†ๆˆ‘ไปฌ็š„่ดก็ŒฎๆŒ‡ๅ—๏ผŒๆˆ‘ไปฌไผšๅฐฝๅฟซๅฎกๆŸฅๅฎƒใ€‚
ๅฆ‚ๆžœๆ‚จ้‡ๅˆฐไปปไฝ•้—ฎ้ข˜๏ผŒ่ฏท้šๆ—ถไธŽๆˆ‘ไปฌ่”็ณปใ€‚

Copy link
Contributor

gru-agent bot commented Apr 17, 2025

TestGru Assignment

Summary

Link CommitId Status Reason
Detail 2c36e86 ๐Ÿšซ Skipped No files need to be tested {"src/libs/agent-runtime/utils/cloudflareHelpers.ts":"target file(src/libs/agent-runtime/utils/cloudflareHelpers.ts) not in work scope \n include: src/utils,src/server,src/database,src/services,src/store,packages//src/ \n exclude: node_modules,/*.test.ts,/.test.tsx,**/.spec.ts,/*.spec.tsx,/.d.ts,**/.test.js,**/*.spec.js,src/database/server"}

Tip

You can @gru-agent and leave your feedback. TestGru will make adjustments based on your input

@dosubot dosubot bot added the ๐Ÿ› Bug Something isn't working | ็ผบ้™ท label Apr 17, 2025
Copy link

codecov bot commented Apr 17, 2025

Codecov Report

All modified and coverable lines are covered by tests โœ…

Project coverage is 89.49%. Comparing base (b6d2c31) to head (2c36e86).
Report is 65 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #7450   +/-   ##
=======================================
  Coverage   89.49%   89.49%           
=======================================
  Files         760      760           
  Lines       54191    54193    +2     
  Branches     3466     3467    +1     
=======================================
+ Hits        48500    48502    +2     
  Misses       5691     5691           
Flag Coverage ฮ”
app 89.49% <100.00%> (+<0.01%) โฌ†๏ธ
server 95.74% <รธ> (รธ)

Flags with carried forward coverage won't be shown. Click here to find out more.

โ˜” View full report in Codecov by Sentry.
๐Ÿ“ข Have feedback on the report? Share it here.

๐Ÿš€ New features to boost your workflow:
  • โ„๏ธ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • ๐Ÿ“ฆ JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@arvinxx arvinxx merged commit 3e81e56 into lobehub:main Apr 24, 2025
27 of 32 checks passed
@lobehubbot
Copy link
Member

โค๏ธ Great PR @BrandonStudio โค๏ธ

The growth of project is inseparable from user feedback and contribution, thanks for your contribution! If you are interesting with the lobehub developer community, please join our discord and then dm @arvinxx or @canisminor1990. They will invite you to our private developer channel. We are talking about the lobe-chat development or sharing ai newsletter around the world.
้กน็›ฎ็š„ๆˆ้•ฟ็ฆปไธๅผ€็”จๆˆทๅ้ฆˆๅ’Œ่ดก็Œฎ๏ผŒๆ„Ÿ่ฐขๆ‚จ็š„่ดก็Œฎ! ๅฆ‚ๆžœๆ‚จๅฏน LobeHub ๅผ€ๅ‘่€…็คพๅŒบๆ„Ÿๅ…ด่ถฃ๏ผŒ่ฏทๅŠ ๅ…ฅๆˆ‘ไปฌ็š„ discord๏ผŒ็„ถๅŽ็งไฟก @arvinxx ๆˆ– @canisminor1990ใ€‚ไป–ไปฌไผš้‚€่ฏทๆ‚จๅŠ ๅ…ฅๆˆ‘ไปฌ็š„็งๅฏ†ๅผ€ๅ‘่€…้ข‘้“ใ€‚ๆˆ‘ไปฌๅฐ†ไผš่ฎจ่ฎบๅ…ณไบŽ Lobe Chat ็š„ๅผ€ๅ‘๏ผŒๅˆ†ไบซๅ’Œ่ฎจ่ฎบๅ…จ็ƒ่Œƒๅ›ดๅ†…็š„ AI ๆถˆๆฏใ€‚

github-actions bot pushed a commit that referenced this pull request Apr 24, 2025
### [Version&nbsp;1.82.6](v1.82.5...v1.82.6)
<sup>Released on **2025-04-24**</sup>

#### ๐Ÿ› Bug Fixes

- **misc**: Improve stability of Cloudflare Workers AI.

<br/>

<details>
<summary><kbd>Improvements and Fixes</kbd></summary>

#### What's fixed

* **misc**: Improve stability of Cloudflare Workers AI, closes [#7450](#7450) ([3e81e56](3e81e56))

</details>

<div align="right">

[![](https://img.shields.io/badge/-BACK_TO_TOP-151515?style=flat-square)](#readme-top)

</div>
@lobehubbot
Copy link
Member

๐ŸŽ‰ This PR is included in version 1.82.6 ๐ŸŽ‰

The release is available on:

Your semantic-release bot ๐Ÿ“ฆ๐Ÿš€

@BrandonStudio BrandonStudio deleted the dev/cf-fix-1 branch April 25, 2025 01:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
๐Ÿ› Bug Something isn't working | ็ผบ้™ท released size:XS This PR changes 0-9 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants