Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: fix the chat stuck in infinite loop #1755

Merged
merged 11 commits into from
Jan 12, 2025

Conversation

zoe27
Copy link
Contributor

@zoe27 zoe27 commented Jan 3, 2025

Relates to:

the chat stuck in infinite loop when using model_local #1213

Risks

Low

Background

What does this PR do?

This PR aim to fix the loop chat of the ai agent self when using the model_local

What kind of change is this?

modify the way of response generate in llama.ts

before this PR:

use the sequence.evaluate to generate the response

in this PR:

adjust to use the chatsession to generate the response

investigation step:

  • using LLAMALOCAL and HEURIST, issue for LLAMALOCAL but HEURIST works wells. which means maybe something wrong with LLAMALOCAL
  • using discord and direct client, HEURIST works well and LLAMALOCAL not well, which means no business with client but on the model
  • https://github.com/withcatai/node-llama-cpp reference to the git and use node-llama-cpp to load local model According to the guidance, it works well, which means the model self is OK
image - write a simple demo to test the model by using the `sequence.evaluate` , loop issue raised - reference to the node-llama-cpp code and adjust the code like in this PR, fix the issue

have not go to deeply to see what is the exactly different between sequence.evaluate and chatsession, maybe can fix loop bug first, and for now, have not found any risk.

Documentation changes needed?

Testing

Where should a reviewer start?

Detailed testing steps

Before
it loop the response and sometimes, the response is hard to understood
image
After
seems in a normal way to chat
image

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @zoe27! Welcome to the ai16z community. Thanks for submitting your first pull request; your efforts are helping us accelerate towards AGI. We'll review it shortly. You are now a ai16z contributor!

@marcellodesales
Copy link
Contributor

@zoe27 The PR is failing with the check against the main branch... Change it to develop to clean the PR checks!

https://github.com/elizaOS/eliza/actions/runs/12597379513/job/35110258487?pr=1755#step:2:8

@zoe27 zoe27 changed the base branch from main to develop January 3, 2025 14:23
@odilitime odilitime changed the title fix the chat stuck in infinite loop fix: fix the chat stuck in infinite loop Jan 3, 2025
@koofree
Copy link
Contributor

koofree commented Jan 7, 2025

@zoe27 Nice! I also figured out this problem, but you have already fixed it! Thanks!

@odilitime odilitime merged commit 35d857e into elizaOS:develop Jan 12, 2025
6 checks passed
mgunnin added a commit to mgunnin/eliza-agent that referenced this pull request Jan 12, 2025
* main: (704 commits)
  bump version (elizaOS#2193)
  feat(security): Implement comprehensive file upload security measures - Add FileSecurityValidator, file type restrictions, size limits, path traversal prevention, enhanced logging and security documentation (elizaOS#1753) (elizaOS#1806)
  fix(client-twitter): clean up mention deduplication (elizaOS#2185)
  fix postgres adapter migration extension creation which already exists at this point (elizaOS#2188)
  Update types.ts
  fix json format typo
  fix quai deps
  fix path
  Add Persian README File
  chore: lint and fix pass on develop (elizaOS#2180)
  bump version to 0,1,8
  bump
  clean up unused var in catch
  comment out unused AkashMessage interface
  bump eslint so it doesn't crash
  remove duplicate TOGETHER in case, lint/unused var
  convert imageDescriptionsArray from let to const per lint
  fix: Koloxarto/fix ragknowledge for postgres (elizaOS#2153)
  fix: fix the chat stuck in infinite loop (elizaOS#1755)
  fix: remove problematic redundant uuid conversion and add api input param validations to api server (elizaOS#2051)
  ...
Tadeuchi added a commit to redstone-finance/eliza that referenced this pull request Jan 13, 2025
0xpi-ai pushed a commit to 0xpi-ai/NayariAI that referenced this pull request Jan 15, 2025
* fix the chat stuck in infinite loop

* perfect the PR and keep the temprature and wordsToPunish in the generate response

* Update README.md

* Trigger CI checks

---------

Co-authored-by: Odilitime <janesmith@airmail.cc>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants