Skip to content
This repository has been archived by the owner on Feb 2, 2025. It is now read-only.

Assistant prefill and stop_sequences async #27

Closed

Conversation

irthomasthomas
Copy link

Changes

  • Updated model implementation to support stop_sequences parameter
  • Added prefill option to prefill Claude's response
  • Updated README with documentation for new features

Testing

  • Verified functionality with claude-3-opus
  • Tested stop sequences and prefill behavior

Notes

Neither the prefill, nor the stop_sequence are included in the response.

llm -m claude-3-opus  "List files sorted by modification time (be brief)" \
-o prefill "<code>" \
-o stop_sequences "</code>,\n"
ls -lt

response_json

"stop_reason": "stop_sequence", "stop_sequence": "</code>"

- Implement prefill option in ClaudeOptions to allow prefilling assistant responses
- Update README with example of using prefill option
- Add test case for prefill functionality
- Implement stop_sequences option for Claude 3 model to allow stopping generation at specified sequences
- Add tests for stop_sequence functionality in both sync and async modes
- Update README with example of using stop_sequences
- Minor refactoring and code cleanup
@irthomasthomas irthomasthomas changed the title Assistant prefill async Assistant prefill and stop_sequences async Nov 19, 2024
@simonw
Copy link
Owner

simonw commented Feb 2, 2025

I really should have re-examined this PR before rolling my own implementation! Sorry about that.

simonw/llm-anthropic@04df4ee

@simonw simonw closed this Feb 2, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants