Skip adding empty content from gemini #768
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
HuanzhiMao
reviewed
Nov 19, 2024
There was a problem hiding this comment.
Hey @gabrielibagon ,
Thanks for the PR.
I think it might be better if we handle it similar to how we do it in the prompting mode?
Because if the model didn't output any response, then in your implementation, the chat history would be like user, model, tool, user, model..., which is not ideal because each tool message should have a model response before the next user query to make it complete; it should be like user, model, tool, model, user, model. I will submit a PR to your branch with my proposed implementation. Let me know what you think.
Handle Situation When Gemini Output No Response in FC Mode
|
@HuanzhiMao That approach looks good, and I agree that it is good to align with the prompting strategy. I've merged your addition to this PR. Thank you! |
HuanzhiMao
added a commit
that referenced
this pull request
Dec 7, 2024
This PR updates the leaderboard to reflect the change in score due to the following PR merge: 1. #747 2. #770 3. #768 4. #750 5. #763 6. #772 7. #777 8. #778 9. #786 10. #787 11. #697 12. #718 13. #755 14. #796 15. #789 16. #804 17. #808 18. #809 19. #811 20. #810 Models were evaluated using checkpoint commit d7e52e5.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR fixes errors where the user adds an empty turn in a multi-turn conversation request, causing the following API error message:
Gemini sometimes returns an empty content during multi-turn conversations, particularly in response to a function result in the previous turn.
As a workaround, we can just skip adding this turn and proceed with the next user turn, which should not affect the correctness of the example unless the missing turn was a function call. But generally, it seems that the missing turn is the optional natural language response to the function response.