Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Custom LLM Support #1079

Merged
merged 5 commits into from
Sep 19, 2024
Merged

Fix Custom LLM Support #1079

merged 5 commits into from
Sep 19, 2024

Conversation

CalebCourier
Copy link
Collaborator

@CalebCourier CalebCourier commented Sep 18, 2024

Fixes: #979

This PR has two main focuses:

  1. Fix support for custom llm_api callables
    • Fix and safeguard when checking if the llm_api is one of OpenAI's static functions
    • Root cause of issue, if OPENAI_API_KEY is not in the environment nor set in an openai global, openai will raise an exception when you try to reference any of the static completion functions even if you're not calling them. We were not properly handling this exception. Now we do.
  2. Method signature enforcement for custom llm_api callables
    • In order to give users upfront feedback, we now inspect custom llm_api callables to ensure they allow for a positional argument for the prompt and accept a spread/rest variable for keyword arguments.
    • We also warn if we do not see keyword arguments for instructions or msg_history since not destructuring out can have unintended consequences when passing **kwargs to any downstream code.

We've also increased test coverage to catch any future regressions to custom llm_api support.

TODO

  • Doc on custom llm_api callables

@CalebCourier CalebCourier requested a review from a team September 18, 2024 20:34
@CalebCourier CalebCourier marked this pull request as draft September 18, 2024 20:41
@CalebCourier CalebCourier marked this pull request as ready for review September 18, 2024 20:58
@CalebCourier CalebCourier marked this pull request as draft September 18, 2024 21:07
@CalebCourier CalebCourier marked this pull request as ready for review September 18, 2024 21:16
@zsimjee zsimjee merged commit eb212ba into main Sep 19, 2024
12 checks passed
@zsimjee zsimjee deleted the custom-llms branch September 19, 2024 22:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[bug] Trouble calling guard without using openai or azure openai APIs
2 participants