Skip to content

Conversation

chuenlok
Copy link

@chuenlok chuenlok commented Nov 20, 2024

What does this PR do?

The assert errors from the server are not returned to the client and hid by the 500 internal server error. Updated the server code (llamastack/llama-stack#493) to send the error and this client can print the error.

  • Addresses issue (#434)

Test Plan

Please describe:

  • tests you ran to verify your changes with result summaries.
  • provide instructions so it can be reproduced.
  1. Kick off the server with llama stack run /Users/henrytai/.llama/distributions/llamastack-ollama/ollama-run.yaml --port 5050
  2. Trigger an error case from client. The example is PYTHONPATH=. python -m examples.agents.client localhost 5050 from the llama-stack-apps repo. The issue details: "This is a known issue with Llama3.2-3B-Instruct on ollama, where the model produces bad tool calling outputs. In this case, the server agent fails due to assert error that "code_interpreter" is not in the AgentConfig's tools field" but the code_interpreter is not really necessary as it is not needed in Llama3.1-8B.

client log:

inference> assistant<|end_header_id|>

Switzerland is a beautiful country with a rich history, stunning mountains, and vibrant cities. Here are three must-visit places in Switzerland:

1. **Geneva**: Geneva is a global hub for international diplomacy and is often referred to as the "City of Peace". It's home to ...

These are just a few examples of Switzerland's amazing attractions!
500: Internal server error: An unexpected error occurred. Tool code_interpreter not found
inference> brave_search.call(query="what is special about #1")...

Showing the real error in the client: Tool code_interpreter not found

Sources

Please link relevant resources if necessary.

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Ran pre-commit to handle lint / formatting issues.
  • Read the contributor guideline,
    Pull Request section?
  • Updated relevant documentation.
  • Wrote necessary unit or integration tests.

@chuenlok
Copy link
Author

@yanxi0830 Can you help review this?

@yanxi0830
Copy link
Contributor

yanxi0830 commented Nov 21, 2024

Thanks! Left some questions. Could you rebase the change on top of current main? There have been some refactors done on the agents lib.

@chuenlok
Copy link
Author

Thanks! Left some questions. Could you rebase the change on top of current main? There have been some refactors done on the agents lib.

Rebased!

@yanxi0830 yanxi0830 merged commit 8805d71 into llamastack:main Nov 22, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants