Litellm dev 11 30 2024 #6974
Merged
Litellm dev 11 30 2024 #6974
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* fix(factory.py): ensure tool call converts image url Fixes #6953 * fix(transformation.py): support mp4 + pdf url's for vertex ai Fixes #6936 * fix(http_handler.py): mask gemini api key in error logs Fixes #6963 * docs(prometheus.md): update prometheus FAQs * feat(auth_checks.py): ensure specific model access > wildcard model access if wildcard model is in access group, but specific model is not - deny access * fix(auth_checks.py): handle auth checks for team based model access groups handles scenario where model access group used for wildcard models * fix(internal_user_endpoints.py): support adding guardrails on `/user/update` Fixes #6942 * fix(key_management_endpoints.py): fix prepare_metadata_fields helper * fix: fix tests * build(requirements.txt): bump openai dep version fixes proxies argument * test: fix tests * fix(http_handler.py): fix error message masking * fix(bedrock_guardrails.py): pass in prepped data * test: fix test * test: fix nvidia nim test * fix(http_handler.py): return original response headers * fix: revert maskedhttpstatuserror * test: update tests * test: cleanup test * fix(key_management_endpoints.py): fix metadata field update logic * fix(key_management_endpoints.py): maintain initial order of guardrails in key update * fix(key_management_endpoints.py): handle prepare metadata * fix: fix linting errors * fix: fix linting errors * fix: fix linting errors * fix: fix key management errors * fix(key_management_endpoints.py): update metadata * test: update test * refactor: add more debug statements * test: skip flaky test * test: fix test * fix: fix test * fix: fix update metadata logic * fix: fix test * ci(config.yml): change db url for e2e ui testing
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
krrishdholakia
added a commit
that referenced
this pull request
Dec 3, 2024
* feat(cohere/chat.py): return citations in model response Closes #6814 * fix(cohere/chat.py): fix linting errors * fix(langsmith.py): support 'run_id' for langsmith Fixes #6862 * fix(langsmith.py): fix langsmith quickstart Fixes #6861 * fix: suppress linting error * LiteLLM Minor Fixes & Improvements (11/29/2024) (#6965) * fix(factory.py): ensure tool call converts image url Fixes #6953 * fix(transformation.py): support mp4 + pdf url's for vertex ai Fixes #6936 * fix(http_handler.py): mask gemini api key in error logs Fixes #6963 * docs(prometheus.md): update prometheus FAQs * feat(auth_checks.py): ensure specific model access > wildcard model access if wildcard model is in access group, but specific model is not - deny access * fix(auth_checks.py): handle auth checks for team based model access groups handles scenario where model access group used for wildcard models * fix(internal_user_endpoints.py): support adding guardrails on `/user/update` Fixes #6942 * fix(key_management_endpoints.py): fix prepare_metadata_fields helper * fix: fix tests * build(requirements.txt): bump openai dep version fixes proxies argument * test: fix tests * fix(http_handler.py): fix error message masking * fix(bedrock_guardrails.py): pass in prepped data * test: fix test * test: fix nvidia nim test * fix(http_handler.py): return original response headers * fix: revert maskedhttpstatuserror * test: update tests * test: cleanup test * fix(key_management_endpoints.py): fix metadata field update logic * fix(key_management_endpoints.py): maintain initial order of guardrails in key update * fix(key_management_endpoints.py): handle prepare metadata * fix: fix linting errors * fix: fix linting errors * fix: fix linting errors * fix: fix key management errors * fix(key_management_endpoints.py): update metadata * test: update test * refactor: add more debug statements * test: skip flaky test * test: fix test * fix: fix test * fix: fix update metadata logic * fix: fix test * ci(config.yml): change db url for e2e ui testing * test: add more debug logs to langsmith * fix: test change * build(config.yml): fix db url '
krrishdholakia
added a commit
that referenced
this pull request
Dec 3, 2024
* add the logprobs param for fireworks ai (#6915) * add the logprobs param for fireworks ai * (feat) pass through llm endpoints - add `PATCH` support (vertex context caching requires for update ops) (#6924) * add PATCH for pass through endpoints * test_pass_through_routes_support_all_methods * sonnet supports pdf, haiku does not (#6928) * (feat) DataDog Logger - Add Failure logging + use Standard Logging payload (#6929) * add async_log_failure_event for dd * use standard logging payload for DD logging * use standard logging payload for DD * fix use SLP status * allow opting into _create_v0_logging_payload * add unit tests for DD logging payload * fix dd logging tests * (feat) log proxy auth errors on datadog (#6931) * add new dd type for auth errors * add async_log_proxy_authentication_errors * fix comment * use async_log_proxy_authentication_errors * test_datadog_post_call_failure_hook * test_async_log_proxy_authentication_errors * (feat) Allow using include to include external YAML files in a config.yaml (#6922) * add helper to process inlcudes directive on yaml * add doc on config management * unit tests for `include` on config.yaml * bump: version 1.52.16 → 1.53. * (feat) dd logger - set tags according to the values set by those env vars (#6933) * dd logger, inherit from .envs * test_datadog_payload_environment_variables * fix _get_datadog_service * build(ui/): update ui build * bump: version 1.53.0 → 1.53.1 * Revert "(feat) Allow using include to include external YAML files in a config.yaml (#6922)" This reverts commit 68e5982. * LiteLLM Minor Fixes & Improvements (11/26/2024) (#6913) * docs(config_settings.md): document all router_settings * ci(config.yml): add router_settings doc test to ci/cd * test: debug test on ci/cd * test: debug ci/cd test * test: fix test * fix(team_endpoints.py): skip invalid team object. don't fail `/team/list` call Causes downstream errors if ui just fails to load team list * test(base_llm_unit_tests.py): add 'response_format={"type": "text"}' test to base_llm_unit_tests adds complete coverage for all 'response_format' values to ci/cd * feat(router.py): support wildcard routes in `get_router_model_info()` Addresses #6914 * build(model_prices_and_context_window.json): add tpm/rpm limits for all gemini models Allows for ratelimit tracking for gemini models even with wildcard routing enabled Addresses #6914 * feat(router.py): add tpm/rpm tracking on success/failure to global_router Addresses #6914 * feat(router.py): support wildcard routes on router.get_model_group_usage() * fix(router.py): fix linting error * fix(router.py): implement get_remaining_tokens_and_requests Addresses #6914 * fix(router.py): fix linting errors * test: fix test * test: fix tests * docs(config_settings.md): add missing dd env vars to docs * fix(router.py): check if hidden params is dict * LiteLLM Minor Fixes & Improvements (11/27/2024) (#6943) * fix(http_parsing_utils.py): remove `ast.literal_eval()` from http utils Security fix - https://huntr.com/bounties/96a32812-213c-4819-ba4e-36143d35e95b?token=bf414bbd77f8b346556e 64ab2dd9301ea44339910877ea50401c76f977e36cdd78272f5fb4ca852a88a7e832828aae1192df98680544ee24aa98f3cf6980d8 bab641a66b7ccbc02c0e7d4ddba2db4dbe7318889dc0098d8db2d639f345f574159814627bb084563bad472e2f990f825bff0878a9 e281e72c88b4bc5884d637d186c0d67c9987c57c3f0caf395aff07b89ad2b7220d1dd7d1b427fd2260b5f01090efce5250f8b56ea2 c0ec19916c24b23825d85ce119911275944c840a1340d69e23ca6a462da610 * fix(converse/transformation.py): support bedrock apac cross region inference Fixes #6905 * fix(user_api_key_auth.py): add auth check for websocket endpoint Fixes #6926 * fix(user_api_key_auth.py): use `model` from query param * fix: fix linting error * test: run flaky tests first * docs: update the docs (#6923) * (bug fix) /key/update was not storing `budget_duration` in the DB (#6941) * fix - store budget_duration for keys * test_generate_and_update_key * test_update_user_unit_test * fix user update * (fix) handle json decode errors for DD exception logging (#6934) * fix JSONDecodeError * handle async_log_proxy_authentication_errors * fix test_async_log_proxy_authentication_errors_get_request * Revert "Revert "(feat) Allow using include to include external YAML files in a config.yaml (#6922)"" This reverts commit 5d13302. * (docs + fix) Add docs on Moderations endpoint, Text Completion (#6947) * fix _pass_through_moderation_endpoint_factory * fix route_llm_request * doc moderations api * docs on /moderations * add e2e tests for moderations api * docs moderations api * test_pass_through_moderation_endpoint_factory * docs text completion * (feat) add enforcement for unique key aliases on /key/update and /key/generate (#6944) * add enforcement for unique key aliases * fix _enforce_unique_key_alias * fix _enforce_unique_key_alias * fix _enforce_unique_key_alias * test_enforce_unique_key_alias * (fix) tag merging / aggregation logic (#6932) * use 1 helper to merge tags + ensure unique ness * test_add_litellm_data_to_request_duplicate_tags * fix _merge_tags * fix proxy utils test * fix doc string * (feat) Allow disabling ErrorLogs written to the DB (#6940) * fix - allow disabling logging error logs * docs on disabling error logs * doc string for _PROXY_failure_handler * test_disable_error_logs * rename file * fix rename file * increase test coverage for test_enable_error_logs * fix(key_management_endpoints.py): support 'tags' param on `/key/update` (#6945) * LiteLLM Minor Fixes & Improvements (11/29/2024) (#6965) * fix(factory.py): ensure tool call converts image url Fixes #6953 * fix(transformation.py): support mp4 + pdf url's for vertex ai Fixes #6936 * fix(http_handler.py): mask gemini api key in error logs Fixes #6963 * docs(prometheus.md): update prometheus FAQs * feat(auth_checks.py): ensure specific model access > wildcard model access if wildcard model is in access group, but specific model is not - deny access * fix(auth_checks.py): handle auth checks for team based model access groups handles scenario where model access group used for wildcard models * fix(internal_user_endpoints.py): support adding guardrails on `/user/update` Fixes #6942 * fix(key_management_endpoints.py): fix prepare_metadata_fields helper * fix: fix tests * build(requirements.txt): bump openai dep version fixes proxies argument * test: fix tests * fix(http_handler.py): fix error message masking * fix(bedrock_guardrails.py): pass in prepped data * test: fix test * test: fix nvidia nim test * fix(http_handler.py): return original response headers * fix: revert maskedhttpstatuserror * test: update tests * test: cleanup test * fix(key_management_endpoints.py): fix metadata field update logic * fix(key_management_endpoints.py): maintain initial order of guardrails in key update * fix(key_management_endpoints.py): handle prepare metadata * fix: fix linting errors * fix: fix linting errors * fix: fix linting errors * fix: fix key management errors * fix(key_management_endpoints.py): update metadata * test: update test * refactor: add more debug statements * test: skip flaky test * test: fix test * fix: fix test * fix: fix update metadata logic * fix: fix test * ci(config.yml): change db url for e2e ui testing * bump: version 1.53.1 → 1.53.2 * Updated config.yml --------- Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: paul-gauthier <69695708+paul-gauthier@users.noreply.github.com> Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com> Co-authored-by: Sara Han <127759186+sdiazlor@users.noreply.github.com> * fix(exceptions.py): ensure ratelimit error code == 429, type == "throttling_error" Fixes #6973 * fix(utils.py): add jina ai dimensions embedding param support Fixes #6591 * fix(exception_mapping_utils.py): add bedrock 'prompt is too long' exception to context window exceeded error exception mapping Fixes #6629 Closes #6975 * feat(databricks/chat): support structured outputs on databricks Closes #6978 - handles content as list for dbrx, - handles streaming+response_format for dbrx * test: fix test * refactor: replace dbrx with 'openai_like' * test: add retry on flaky test * fix(main.py): fix vertex meta llama api call * docs(json_mode.md): update json docs * Update team_endpoints.py (#6983) correct the url what should be member_delete and not update * fix: trigger new build * Litellm dev 11 30 2024 (#6974) * feat(cohere/chat.py): return citations in model response Closes #6814 * fix(cohere/chat.py): fix linting errors * fix(langsmith.py): support 'run_id' for langsmith Fixes #6862 * fix(langsmith.py): fix langsmith quickstart Fixes #6861 * fix: suppress linting error * LiteLLM Minor Fixes & Improvements (11/29/2024) (#6965) * fix(factory.py): ensure tool call converts image url Fixes #6953 * fix(transformation.py): support mp4 + pdf url's for vertex ai Fixes #6936 * fix(http_handler.py): mask gemini api key in error logs Fixes #6963 * docs(prometheus.md): update prometheus FAQs * feat(auth_checks.py): ensure specific model access > wildcard model access if wildcard model is in access group, but specific model is not - deny access * fix(auth_checks.py): handle auth checks for team based model access groups handles scenario where model access group used for wildcard models * fix(internal_user_endpoints.py): support adding guardrails on `/user/update` Fixes #6942 * fix(key_management_endpoints.py): fix prepare_metadata_fields helper * fix: fix tests * build(requirements.txt): bump openai dep version fixes proxies argument * test: fix tests * fix(http_handler.py): fix error message masking * fix(bedrock_guardrails.py): pass in prepped data * test: fix test * test: fix nvidia nim test * fix(http_handler.py): return original response headers * fix: revert maskedhttpstatuserror * test: update tests * test: cleanup test * fix(key_management_endpoints.py): fix metadata field update logic * fix(key_management_endpoints.py): maintain initial order of guardrails in key update * fix(key_management_endpoints.py): handle prepare metadata * fix: fix linting errors * fix: fix linting errors * fix: fix linting errors * fix: fix key management errors * fix(key_management_endpoints.py): update metadata * test: update test * refactor: add more debug statements * test: skip flaky test * test: fix test * fix: fix test * fix: fix update metadata logic * fix: fix test * ci(config.yml): change db url for e2e ui testing * test: add more debug logs to langsmith * fix: test change * build(config.yml): fix db url ' * Litellm 12 02 2024 (#6994) * add the logprobs param for fireworks ai (#6915) * add the logprobs param for fireworks ai * (feat) pass through llm endpoints - add `PATCH` support (vertex context caching requires for update ops) (#6924) * add PATCH for pass through endpoints * test_pass_through_routes_support_all_methods * sonnet supports pdf, haiku does not (#6928) * (feat) DataDog Logger - Add Failure logging + use Standard Logging payload (#6929) * add async_log_failure_event for dd * use standard logging payload for DD logging * use standard logging payload for DD * fix use SLP status * allow opting into _create_v0_logging_payload * add unit tests for DD logging payload * fix dd logging tests * (feat) log proxy auth errors on datadog (#6931) * add new dd type for auth errors * add async_log_proxy_authentication_errors * fix comment * use async_log_proxy_authentication_errors * test_datadog_post_call_failure_hook * test_async_log_proxy_authentication_errors * (feat) Allow using include to include external YAML files in a config.yaml (#6922) * add helper to process inlcudes directive on yaml * add doc on config management * unit tests for `include` on config.yaml * bump: version 1.52.16 → 1.53. * (feat) dd logger - set tags according to the values set by those env vars (#6933) * dd logger, inherit from .envs * test_datadog_payload_environment_variables * fix _get_datadog_service * build(ui/): update ui build * bump: version 1.53.0 → 1.53.1 * Revert "(feat) Allow using include to include external YAML files in a config.yaml (#6922)" This reverts commit 68e5982. * LiteLLM Minor Fixes & Improvements (11/26/2024) (#6913) * docs(config_settings.md): document all router_settings * ci(config.yml): add router_settings doc test to ci/cd * test: debug test on ci/cd * test: debug ci/cd test * test: fix test * fix(team_endpoints.py): skip invalid team object. don't fail `/team/list` call Causes downstream errors if ui just fails to load team list * test(base_llm_unit_tests.py): add 'response_format={"type": "text"}' test to base_llm_unit_tests adds complete coverage for all 'response_format' values to ci/cd * feat(router.py): support wildcard routes in `get_router_model_info()` Addresses #6914 * build(model_prices_and_context_window.json): add tpm/rpm limits for all gemini models Allows for ratelimit tracking for gemini models even with wildcard routing enabled Addresses #6914 * feat(router.py): add tpm/rpm tracking on success/failure to global_router Addresses #6914 * feat(router.py): support wildcard routes on router.get_model_group_usage() * fix(router.py): fix linting error * fix(router.py): implement get_remaining_tokens_and_requests Addresses #6914 * fix(router.py): fix linting errors * test: fix test * test: fix tests * docs(config_settings.md): add missing dd env vars to docs * fix(router.py): check if hidden params is dict * LiteLLM Minor Fixes & Improvements (11/27/2024) (#6943) * fix(http_parsing_utils.py): remove `ast.literal_eval()` from http utils Security fix - https://huntr.com/bounties/96a32812-213c-4819-ba4e-36143d35e95b?token=bf414bbd77f8b346556e 64ab2dd9301ea44339910877ea50401c76f977e36cdd78272f5fb4ca852a88a7e832828aae1192df98680544ee24aa98f3cf6980d8 bab641a66b7ccbc02c0e7d4ddba2db4dbe7318889dc0098d8db2d639f345f574159814627bb084563bad472e2f990f825bff0878a9 e281e72c88b4bc5884d637d186c0d67c9987c57c3f0caf395aff07b89ad2b7220d1dd7d1b427fd2260b5f01090efce5250f8b56ea2 c0ec19916c24b23825d85ce119911275944c840a1340d69e23ca6a462da610 * fix(converse/transformation.py): support bedrock apac cross region inference Fixes #6905 * fix(user_api_key_auth.py): add auth check for websocket endpoint Fixes #6926 * fix(user_api_key_auth.py): use `model` from query param * fix: fix linting error * test: run flaky tests first * docs: update the docs (#6923) * (bug fix) /key/update was not storing `budget_duration` in the DB (#6941) * fix - store budget_duration for keys * test_generate_and_update_key * test_update_user_unit_test * fix user update * (fix) handle json decode errors for DD exception logging (#6934) * fix JSONDecodeError * handle async_log_proxy_authentication_errors * fix test_async_log_proxy_authentication_errors_get_request * Revert "Revert "(feat) Allow using include to include external YAML files in a config.yaml (#6922)"" This reverts commit 5d13302. * (docs + fix) Add docs on Moderations endpoint, Text Completion (#6947) * fix _pass_through_moderation_endpoint_factory * fix route_llm_request * doc moderations api * docs on /moderations * add e2e tests for moderations api * docs moderations api * test_pass_through_moderation_endpoint_factory * docs text completion * (feat) add enforcement for unique key aliases on /key/update and /key/generate (#6944) * add enforcement for unique key aliases * fix _enforce_unique_key_alias * fix _enforce_unique_key_alias * fix _enforce_unique_key_alias * test_enforce_unique_key_alias * (fix) tag merging / aggregation logic (#6932) * use 1 helper to merge tags + ensure unique ness * test_add_litellm_data_to_request_duplicate_tags * fix _merge_tags * fix proxy utils test * fix doc string * (feat) Allow disabling ErrorLogs written to the DB (#6940) * fix - allow disabling logging error logs * docs on disabling error logs * doc string for _PROXY_failure_handler * test_disable_error_logs * rename file * fix rename file * increase test coverage for test_enable_error_logs * fix(key_management_endpoints.py): support 'tags' param on `/key/update` (#6945) * LiteLLM Minor Fixes & Improvements (11/29/2024) (#6965) * fix(factory.py): ensure tool call converts image url Fixes #6953 * fix(transformation.py): support mp4 + pdf url's for vertex ai Fixes #6936 * fix(http_handler.py): mask gemini api key in error logs Fixes #6963 * docs(prometheus.md): update prometheus FAQs * feat(auth_checks.py): ensure specific model access > wildcard model access if wildcard model is in access group, but specific model is not - deny access * fix(auth_checks.py): handle auth checks for team based model access groups handles scenario where model access group used for wildcard models * fix(internal_user_endpoints.py): support adding guardrails on `/user/update` Fixes #6942 * fix(key_management_endpoints.py): fix prepare_metadata_fields helper * fix: fix tests * build(requirements.txt): bump openai dep version fixes proxies argument * test: fix tests * fix(http_handler.py): fix error message masking * fix(bedrock_guardrails.py): pass in prepped data * test: fix test * test: fix nvidia nim test * fix(http_handler.py): return original response headers * fix: revert maskedhttpstatuserror * test: update tests * test: cleanup test * fix(key_management_endpoints.py): fix metadata field update logic * fix(key_management_endpoints.py): maintain initial order of guardrails in key update * fix(key_management_endpoints.py): handle prepare metadata * fix: fix linting errors * fix: fix linting errors * fix: fix linting errors * fix: fix key management errors * fix(key_management_endpoints.py): update metadata * test: update test * refactor: add more debug statements * test: skip flaky test * test: fix test * fix: fix test * fix: fix update metadata logic * fix: fix test * ci(config.yml): change db url for e2e ui testing * bump: version 1.53.1 → 1.53.2 * Updated config.yml --------- Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: paul-gauthier <69695708+paul-gauthier@users.noreply.github.com> Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com> Co-authored-by: Sara Han <127759186+sdiazlor@users.noreply.github.com> * fix(exceptions.py): ensure ratelimit error code == 429, type == "throttling_error" Fixes #6973 * fix(utils.py): add jina ai dimensions embedding param support Fixes #6591 * fix(exception_mapping_utils.py): add bedrock 'prompt is too long' exception to context window exceeded error exception mapping Fixes #6629 Closes #6975 * fix(litellm_logging.py): strip trailing slash for api base Closes #6859 * test: skip timeout issue --------- Co-authored-by: ershang-dou <erlie.shang@gmail.com> Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: paul-gauthier <69695708+paul-gauthier@users.noreply.github.com> Co-authored-by: Sara Han <127759186+sdiazlor@users.noreply.github.com> * Litellm test ci cd (#6997) * ci(config.yml): run parallel tests * fix(conftest.py): add pytest retry summary * ci(config.yml;): add pip install "pytest-xdist==3.6.1" * ci: revert conftest * (fix) logging Auth errors on datadog (#6995) * fix get_standard_logging_object_payload * fix async_post_call_failure_hook * fix post_call_failure_hook * fix change * fix _is_proxy_only_error * fix async_post_call_failure_hook * fix getting request body * remove redundant code * use a well named original function name for auth errors * fix logging auth fails on DD * fix using request body * use helper for _handle_logging_proxy_only_error * (fixes) datadog logging - handle 1MB max log size on DD (#6996) * fix dd truncate_standard_logging_payload_content * dd truncate_standard_logging_payload_content * fix test_datadog_payload_content_truncation * add clear msg on _truncate_text * test_truncate_standard_logging_payload * fix linting error * fix linting errors * bump: version 1.53.2 → 1.53.3 --------- Co-authored-by: ershang-dou <erlie.shang@gmail.com> Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: paul-gauthier <69695708+paul-gauthier@users.noreply.github.com> Co-authored-by: Sara Han <127759186+sdiazlor@users.noreply.github.com> Co-authored-by: superpoussin22 <vincent.nadal@orange.fr>
rajatvig
pushed a commit
to rajatvig/litellm
that referenced
this pull request
Jan 16, 2025
* feat(cohere/chat.py): return citations in model response Closes BerriAI#6814 * fix(cohere/chat.py): fix linting errors * fix(langsmith.py): support 'run_id' for langsmith Fixes BerriAI#6862 * fix(langsmith.py): fix langsmith quickstart Fixes BerriAI#6861 * fix: suppress linting error * LiteLLM Minor Fixes & Improvements (11/29/2024) (BerriAI#6965) * fix(factory.py): ensure tool call converts image url Fixes BerriAI#6953 * fix(transformation.py): support mp4 + pdf url's for vertex ai Fixes BerriAI#6936 * fix(http_handler.py): mask gemini api key in error logs Fixes BerriAI#6963 * docs(prometheus.md): update prometheus FAQs * feat(auth_checks.py): ensure specific model access > wildcard model access if wildcard model is in access group, but specific model is not - deny access * fix(auth_checks.py): handle auth checks for team based model access groups handles scenario where model access group used for wildcard models * fix(internal_user_endpoints.py): support adding guardrails on `/user/update` Fixes BerriAI#6942 * fix(key_management_endpoints.py): fix prepare_metadata_fields helper * fix: fix tests * build(requirements.txt): bump openai dep version fixes proxies argument * test: fix tests * fix(http_handler.py): fix error message masking * fix(bedrock_guardrails.py): pass in prepped data * test: fix test * test: fix nvidia nim test * fix(http_handler.py): return original response headers * fix: revert maskedhttpstatuserror * test: update tests * test: cleanup test * fix(key_management_endpoints.py): fix metadata field update logic * fix(key_management_endpoints.py): maintain initial order of guardrails in key update * fix(key_management_endpoints.py): handle prepare metadata * fix: fix linting errors * fix: fix linting errors * fix: fix linting errors * fix: fix key management errors * fix(key_management_endpoints.py): update metadata * test: update test * refactor: add more debug statements * test: skip flaky test * test: fix test * fix: fix test * fix: fix update metadata logic * fix: fix test * ci(config.yml): change db url for e2e ui testing * test: add more debug logs to langsmith * fix: test change * build(config.yml): fix db url '
rajatvig
pushed a commit
to rajatvig/litellm
that referenced
this pull request
Jan 16, 2025
* add the logprobs param for fireworks ai (BerriAI#6915) * add the logprobs param for fireworks ai * (feat) pass through llm endpoints - add `PATCH` support (vertex context caching requires for update ops) (BerriAI#6924) * add PATCH for pass through endpoints * test_pass_through_routes_support_all_methods * sonnet supports pdf, haiku does not (BerriAI#6928) * (feat) DataDog Logger - Add Failure logging + use Standard Logging payload (BerriAI#6929) * add async_log_failure_event for dd * use standard logging payload for DD logging * use standard logging payload for DD * fix use SLP status * allow opting into _create_v0_logging_payload * add unit tests for DD logging payload * fix dd logging tests * (feat) log proxy auth errors on datadog (BerriAI#6931) * add new dd type for auth errors * add async_log_proxy_authentication_errors * fix comment * use async_log_proxy_authentication_errors * test_datadog_post_call_failure_hook * test_async_log_proxy_authentication_errors * (feat) Allow using include to include external YAML files in a config.yaml (BerriAI#6922) * add helper to process inlcudes directive on yaml * add doc on config management * unit tests for `include` on config.yaml * bump: version 1.52.16 → 1.53. * (feat) dd logger - set tags according to the values set by those env vars (BerriAI#6933) * dd logger, inherit from .envs * test_datadog_payload_environment_variables * fix _get_datadog_service * build(ui/): update ui build * bump: version 1.53.0 → 1.53.1 * Revert "(feat) Allow using include to include external YAML files in a config.yaml (BerriAI#6922)" This reverts commit 68e5982. * LiteLLM Minor Fixes & Improvements (11/26/2024) (BerriAI#6913) * docs(config_settings.md): document all router_settings * ci(config.yml): add router_settings doc test to ci/cd * test: debug test on ci/cd * test: debug ci/cd test * test: fix test * fix(team_endpoints.py): skip invalid team object. don't fail `/team/list` call Causes downstream errors if ui just fails to load team list * test(base_llm_unit_tests.py): add 'response_format={"type": "text"}' test to base_llm_unit_tests adds complete coverage for all 'response_format' values to ci/cd * feat(router.py): support wildcard routes in `get_router_model_info()` Addresses BerriAI#6914 * build(model_prices_and_context_window.json): add tpm/rpm limits for all gemini models Allows for ratelimit tracking for gemini models even with wildcard routing enabled Addresses BerriAI#6914 * feat(router.py): add tpm/rpm tracking on success/failure to global_router Addresses BerriAI#6914 * feat(router.py): support wildcard routes on router.get_model_group_usage() * fix(router.py): fix linting error * fix(router.py): implement get_remaining_tokens_and_requests Addresses BerriAI#6914 * fix(router.py): fix linting errors * test: fix test * test: fix tests * docs(config_settings.md): add missing dd env vars to docs * fix(router.py): check if hidden params is dict * LiteLLM Minor Fixes & Improvements (11/27/2024) (BerriAI#6943) * fix(http_parsing_utils.py): remove `ast.literal_eval()` from http utils Security fix - https://huntr.com/bounties/96a32812-213c-4819-ba4e-36143d35e95b?token=bf414bbd77f8b346556e 64ab2dd9301ea44339910877ea50401c76f977e36cdd78272f5fb4ca852a88a7e832828aae1192df98680544ee24aa98f3cf6980d8 bab641a66b7ccbc02c0e7d4ddba2db4dbe7318889dc0098d8db2d639f345f574159814627bb084563bad472e2f990f825bff0878a9 e281e72c88b4bc5884d637d186c0d67c9987c57c3f0caf395aff07b89ad2b7220d1dd7d1b427fd2260b5f01090efce5250f8b56ea2 c0ec19916c24b23825d85ce119911275944c840a1340d69e23ca6a462da610 * fix(converse/transformation.py): support bedrock apac cross region inference Fixes BerriAI#6905 * fix(user_api_key_auth.py): add auth check for websocket endpoint Fixes BerriAI#6926 * fix(user_api_key_auth.py): use `model` from query param * fix: fix linting error * test: run flaky tests first * docs: update the docs (BerriAI#6923) * (bug fix) /key/update was not storing `budget_duration` in the DB (BerriAI#6941) * fix - store budget_duration for keys * test_generate_and_update_key * test_update_user_unit_test * fix user update * (fix) handle json decode errors for DD exception logging (BerriAI#6934) * fix JSONDecodeError * handle async_log_proxy_authentication_errors * fix test_async_log_proxy_authentication_errors_get_request * Revert "Revert "(feat) Allow using include to include external YAML files in a config.yaml (BerriAI#6922)"" This reverts commit 5d13302. * (docs + fix) Add docs on Moderations endpoint, Text Completion (BerriAI#6947) * fix _pass_through_moderation_endpoint_factory * fix route_llm_request * doc moderations api * docs on /moderations * add e2e tests for moderations api * docs moderations api * test_pass_through_moderation_endpoint_factory * docs text completion * (feat) add enforcement for unique key aliases on /key/update and /key/generate (BerriAI#6944) * add enforcement for unique key aliases * fix _enforce_unique_key_alias * fix _enforce_unique_key_alias * fix _enforce_unique_key_alias * test_enforce_unique_key_alias * (fix) tag merging / aggregation logic (BerriAI#6932) * use 1 helper to merge tags + ensure unique ness * test_add_litellm_data_to_request_duplicate_tags * fix _merge_tags * fix proxy utils test * fix doc string * (feat) Allow disabling ErrorLogs written to the DB (BerriAI#6940) * fix - allow disabling logging error logs * docs on disabling error logs * doc string for _PROXY_failure_handler * test_disable_error_logs * rename file * fix rename file * increase test coverage for test_enable_error_logs * fix(key_management_endpoints.py): support 'tags' param on `/key/update` (BerriAI#6945) * LiteLLM Minor Fixes & Improvements (11/29/2024) (BerriAI#6965) * fix(factory.py): ensure tool call converts image url Fixes BerriAI#6953 * fix(transformation.py): support mp4 + pdf url's for vertex ai Fixes BerriAI#6936 * fix(http_handler.py): mask gemini api key in error logs Fixes BerriAI#6963 * docs(prometheus.md): update prometheus FAQs * feat(auth_checks.py): ensure specific model access > wildcard model access if wildcard model is in access group, but specific model is not - deny access * fix(auth_checks.py): handle auth checks for team based model access groups handles scenario where model access group used for wildcard models * fix(internal_user_endpoints.py): support adding guardrails on `/user/update` Fixes BerriAI#6942 * fix(key_management_endpoints.py): fix prepare_metadata_fields helper * fix: fix tests * build(requirements.txt): bump openai dep version fixes proxies argument * test: fix tests * fix(http_handler.py): fix error message masking * fix(bedrock_guardrails.py): pass in prepped data * test: fix test * test: fix nvidia nim test * fix(http_handler.py): return original response headers * fix: revert maskedhttpstatuserror * test: update tests * test: cleanup test * fix(key_management_endpoints.py): fix metadata field update logic * fix(key_management_endpoints.py): maintain initial order of guardrails in key update * fix(key_management_endpoints.py): handle prepare metadata * fix: fix linting errors * fix: fix linting errors * fix: fix linting errors * fix: fix key management errors * fix(key_management_endpoints.py): update metadata * test: update test * refactor: add more debug statements * test: skip flaky test * test: fix test * fix: fix test * fix: fix update metadata logic * fix: fix test * ci(config.yml): change db url for e2e ui testing * bump: version 1.53.1 → 1.53.2 * Updated config.yml --------- Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: paul-gauthier <69695708+paul-gauthier@users.noreply.github.com> Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com> Co-authored-by: Sara Han <127759186+sdiazlor@users.noreply.github.com> * fix(exceptions.py): ensure ratelimit error code == 429, type == "throttling_error" Fixes BerriAI#6973 * fix(utils.py): add jina ai dimensions embedding param support Fixes BerriAI#6591 * fix(exception_mapping_utils.py): add bedrock 'prompt is too long' exception to context window exceeded error exception mapping Fixes BerriAI#6629 Closes BerriAI#6975 * feat(databricks/chat): support structured outputs on databricks Closes BerriAI#6978 - handles content as list for dbrx, - handles streaming+response_format for dbrx * test: fix test * refactor: replace dbrx with 'openai_like' * test: add retry on flaky test * fix(main.py): fix vertex meta llama api call * docs(json_mode.md): update json docs * Update team_endpoints.py (BerriAI#6983) correct the url what should be member_delete and not update * fix: trigger new build * Litellm dev 11 30 2024 (BerriAI#6974) * feat(cohere/chat.py): return citations in model response Closes BerriAI#6814 * fix(cohere/chat.py): fix linting errors * fix(langsmith.py): support 'run_id' for langsmith Fixes BerriAI#6862 * fix(langsmith.py): fix langsmith quickstart Fixes BerriAI#6861 * fix: suppress linting error * LiteLLM Minor Fixes & Improvements (11/29/2024) (BerriAI#6965) * fix(factory.py): ensure tool call converts image url Fixes BerriAI#6953 * fix(transformation.py): support mp4 + pdf url's for vertex ai Fixes BerriAI#6936 * fix(http_handler.py): mask gemini api key in error logs Fixes BerriAI#6963 * docs(prometheus.md): update prometheus FAQs * feat(auth_checks.py): ensure specific model access > wildcard model access if wildcard model is in access group, but specific model is not - deny access * fix(auth_checks.py): handle auth checks for team based model access groups handles scenario where model access group used for wildcard models * fix(internal_user_endpoints.py): support adding guardrails on `/user/update` Fixes BerriAI#6942 * fix(key_management_endpoints.py): fix prepare_metadata_fields helper * fix: fix tests * build(requirements.txt): bump openai dep version fixes proxies argument * test: fix tests * fix(http_handler.py): fix error message masking * fix(bedrock_guardrails.py): pass in prepped data * test: fix test * test: fix nvidia nim test * fix(http_handler.py): return original response headers * fix: revert maskedhttpstatuserror * test: update tests * test: cleanup test * fix(key_management_endpoints.py): fix metadata field update logic * fix(key_management_endpoints.py): maintain initial order of guardrails in key update * fix(key_management_endpoints.py): handle prepare metadata * fix: fix linting errors * fix: fix linting errors * fix: fix linting errors * fix: fix key management errors * fix(key_management_endpoints.py): update metadata * test: update test * refactor: add more debug statements * test: skip flaky test * test: fix test * fix: fix test * fix: fix update metadata logic * fix: fix test * ci(config.yml): change db url for e2e ui testing * test: add more debug logs to langsmith * fix: test change * build(config.yml): fix db url ' * Litellm 12 02 2024 (BerriAI#6994) * add the logprobs param for fireworks ai (BerriAI#6915) * add the logprobs param for fireworks ai * (feat) pass through llm endpoints - add `PATCH` support (vertex context caching requires for update ops) (BerriAI#6924) * add PATCH for pass through endpoints * test_pass_through_routes_support_all_methods * sonnet supports pdf, haiku does not (BerriAI#6928) * (feat) DataDog Logger - Add Failure logging + use Standard Logging payload (BerriAI#6929) * add async_log_failure_event for dd * use standard logging payload for DD logging * use standard logging payload for DD * fix use SLP status * allow opting into _create_v0_logging_payload * add unit tests for DD logging payload * fix dd logging tests * (feat) log proxy auth errors on datadog (BerriAI#6931) * add new dd type for auth errors * add async_log_proxy_authentication_errors * fix comment * use async_log_proxy_authentication_errors * test_datadog_post_call_failure_hook * test_async_log_proxy_authentication_errors * (feat) Allow using include to include external YAML files in a config.yaml (BerriAI#6922) * add helper to process inlcudes directive on yaml * add doc on config management * unit tests for `include` on config.yaml * bump: version 1.52.16 → 1.53. * (feat) dd logger - set tags according to the values set by those env vars (BerriAI#6933) * dd logger, inherit from .envs * test_datadog_payload_environment_variables * fix _get_datadog_service * build(ui/): update ui build * bump: version 1.53.0 → 1.53.1 * Revert "(feat) Allow using include to include external YAML files in a config.yaml (BerriAI#6922)" This reverts commit 68e5982. * LiteLLM Minor Fixes & Improvements (11/26/2024) (BerriAI#6913) * docs(config_settings.md): document all router_settings * ci(config.yml): add router_settings doc test to ci/cd * test: debug test on ci/cd * test: debug ci/cd test * test: fix test * fix(team_endpoints.py): skip invalid team object. don't fail `/team/list` call Causes downstream errors if ui just fails to load team list * test(base_llm_unit_tests.py): add 'response_format={"type": "text"}' test to base_llm_unit_tests adds complete coverage for all 'response_format' values to ci/cd * feat(router.py): support wildcard routes in `get_router_model_info()` Addresses BerriAI#6914 * build(model_prices_and_context_window.json): add tpm/rpm limits for all gemini models Allows for ratelimit tracking for gemini models even with wildcard routing enabled Addresses BerriAI#6914 * feat(router.py): add tpm/rpm tracking on success/failure to global_router Addresses BerriAI#6914 * feat(router.py): support wildcard routes on router.get_model_group_usage() * fix(router.py): fix linting error * fix(router.py): implement get_remaining_tokens_and_requests Addresses BerriAI#6914 * fix(router.py): fix linting errors * test: fix test * test: fix tests * docs(config_settings.md): add missing dd env vars to docs * fix(router.py): check if hidden params is dict * LiteLLM Minor Fixes & Improvements (11/27/2024) (BerriAI#6943) * fix(http_parsing_utils.py): remove `ast.literal_eval()` from http utils Security fix - https://huntr.com/bounties/96a32812-213c-4819-ba4e-36143d35e95b?token=bf414bbd77f8b346556e 64ab2dd9301ea44339910877ea50401c76f977e36cdd78272f5fb4ca852a88a7e832828aae1192df98680544ee24aa98f3cf6980d8 bab641a66b7ccbc02c0e7d4ddba2db4dbe7318889dc0098d8db2d639f345f574159814627bb084563bad472e2f990f825bff0878a9 e281e72c88b4bc5884d637d186c0d67c9987c57c3f0caf395aff07b89ad2b7220d1dd7d1b427fd2260b5f01090efce5250f8b56ea2 c0ec19916c24b23825d85ce119911275944c840a1340d69e23ca6a462da610 * fix(converse/transformation.py): support bedrock apac cross region inference Fixes BerriAI#6905 * fix(user_api_key_auth.py): add auth check for websocket endpoint Fixes BerriAI#6926 * fix(user_api_key_auth.py): use `model` from query param * fix: fix linting error * test: run flaky tests first * docs: update the docs (BerriAI#6923) * (bug fix) /key/update was not storing `budget_duration` in the DB (BerriAI#6941) * fix - store budget_duration for keys * test_generate_and_update_key * test_update_user_unit_test * fix user update * (fix) handle json decode errors for DD exception logging (BerriAI#6934) * fix JSONDecodeError * handle async_log_proxy_authentication_errors * fix test_async_log_proxy_authentication_errors_get_request * Revert "Revert "(feat) Allow using include to include external YAML files in a config.yaml (BerriAI#6922)"" This reverts commit 5d13302. * (docs + fix) Add docs on Moderations endpoint, Text Completion (BerriAI#6947) * fix _pass_through_moderation_endpoint_factory * fix route_llm_request * doc moderations api * docs on /moderations * add e2e tests for moderations api * docs moderations api * test_pass_through_moderation_endpoint_factory * docs text completion * (feat) add enforcement for unique key aliases on /key/update and /key/generate (BerriAI#6944) * add enforcement for unique key aliases * fix _enforce_unique_key_alias * fix _enforce_unique_key_alias * fix _enforce_unique_key_alias * test_enforce_unique_key_alias * (fix) tag merging / aggregation logic (BerriAI#6932) * use 1 helper to merge tags + ensure unique ness * test_add_litellm_data_to_request_duplicate_tags * fix _merge_tags * fix proxy utils test * fix doc string * (feat) Allow disabling ErrorLogs written to the DB (BerriAI#6940) * fix - allow disabling logging error logs * docs on disabling error logs * doc string for _PROXY_failure_handler * test_disable_error_logs * rename file * fix rename file * increase test coverage for test_enable_error_logs * fix(key_management_endpoints.py): support 'tags' param on `/key/update` (BerriAI#6945) * LiteLLM Minor Fixes & Improvements (11/29/2024) (BerriAI#6965) * fix(factory.py): ensure tool call converts image url Fixes BerriAI#6953 * fix(transformation.py): support mp4 + pdf url's for vertex ai Fixes BerriAI#6936 * fix(http_handler.py): mask gemini api key in error logs Fixes BerriAI#6963 * docs(prometheus.md): update prometheus FAQs * feat(auth_checks.py): ensure specific model access > wildcard model access if wildcard model is in access group, but specific model is not - deny access * fix(auth_checks.py): handle auth checks for team based model access groups handles scenario where model access group used for wildcard models * fix(internal_user_endpoints.py): support adding guardrails on `/user/update` Fixes BerriAI#6942 * fix(key_management_endpoints.py): fix prepare_metadata_fields helper * fix: fix tests * build(requirements.txt): bump openai dep version fixes proxies argument * test: fix tests * fix(http_handler.py): fix error message masking * fix(bedrock_guardrails.py): pass in prepped data * test: fix test * test: fix nvidia nim test * fix(http_handler.py): return original response headers * fix: revert maskedhttpstatuserror * test: update tests * test: cleanup test * fix(key_management_endpoints.py): fix metadata field update logic * fix(key_management_endpoints.py): maintain initial order of guardrails in key update * fix(key_management_endpoints.py): handle prepare metadata * fix: fix linting errors * fix: fix linting errors * fix: fix linting errors * fix: fix key management errors * fix(key_management_endpoints.py): update metadata * test: update test * refactor: add more debug statements * test: skip flaky test * test: fix test * fix: fix test * fix: fix update metadata logic * fix: fix test * ci(config.yml): change db url for e2e ui testing * bump: version 1.53.1 → 1.53.2 * Updated config.yml --------- Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: paul-gauthier <69695708+paul-gauthier@users.noreply.github.com> Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com> Co-authored-by: Sara Han <127759186+sdiazlor@users.noreply.github.com> * fix(exceptions.py): ensure ratelimit error code == 429, type == "throttling_error" Fixes BerriAI#6973 * fix(utils.py): add jina ai dimensions embedding param support Fixes BerriAI#6591 * fix(exception_mapping_utils.py): add bedrock 'prompt is too long' exception to context window exceeded error exception mapping Fixes BerriAI#6629 Closes BerriAI#6975 * fix(litellm_logging.py): strip trailing slash for api base Closes BerriAI#6859 * test: skip timeout issue --------- Co-authored-by: ershang-dou <erlie.shang@gmail.com> Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: paul-gauthier <69695708+paul-gauthier@users.noreply.github.com> Co-authored-by: Sara Han <127759186+sdiazlor@users.noreply.github.com> * Litellm test ci cd (BerriAI#6997) * ci(config.yml): run parallel tests * fix(conftest.py): add pytest retry summary * ci(config.yml;): add pip install "pytest-xdist==3.6.1" * ci: revert conftest * (fix) logging Auth errors on datadog (BerriAI#6995) * fix get_standard_logging_object_payload * fix async_post_call_failure_hook * fix post_call_failure_hook * fix change * fix _is_proxy_only_error * fix async_post_call_failure_hook * fix getting request body * remove redundant code * use a well named original function name for auth errors * fix logging auth fails on DD * fix using request body * use helper for _handle_logging_proxy_only_error * (fixes) datadog logging - handle 1MB max log size on DD (BerriAI#6996) * fix dd truncate_standard_logging_payload_content * dd truncate_standard_logging_payload_content * fix test_datadog_payload_content_truncation * add clear msg on _truncate_text * test_truncate_standard_logging_payload * fix linting error * fix linting errors * bump: version 1.53.2 → 1.53.3 --------- Co-authored-by: ershang-dou <erlie.shang@gmail.com> Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: paul-gauthier <69695708+paul-gauthier@users.noreply.github.com> Co-authored-by: Sara Han <127759186+sdiazlor@users.noreply.github.com> Co-authored-by: superpoussin22 <vincent.nadal@orange.fr>
rajatvig
pushed a commit
to rajatvig/litellm
that referenced
this pull request
Jan 16, 2025
* add the logprobs param for fireworks ai (BerriAI#6915) * add the logprobs param for fireworks ai * (feat) pass through llm endpoints - add `PATCH` support (vertex context caching requires for update ops) (BerriAI#6924) * add PATCH for pass through endpoints * test_pass_through_routes_support_all_methods * sonnet supports pdf, haiku does not (BerriAI#6928) * (feat) DataDog Logger - Add Failure logging + use Standard Logging payload (BerriAI#6929) * add async_log_failure_event for dd * use standard logging payload for DD logging * use standard logging payload for DD * fix use SLP status * allow opting into _create_v0_logging_payload * add unit tests for DD logging payload * fix dd logging tests * (feat) log proxy auth errors on datadog (BerriAI#6931) * add new dd type for auth errors * add async_log_proxy_authentication_errors * fix comment * use async_log_proxy_authentication_errors * test_datadog_post_call_failure_hook * test_async_log_proxy_authentication_errors * (feat) Allow using include to include external YAML files in a config.yaml (BerriAI#6922) * add helper to process inlcudes directive on yaml * add doc on config management * unit tests for `include` on config.yaml * bump: version 1.52.16 → 1.53. * (feat) dd logger - set tags according to the values set by those env vars (BerriAI#6933) * dd logger, inherit from .envs * test_datadog_payload_environment_variables * fix _get_datadog_service * build(ui/): update ui build * bump: version 1.53.0 → 1.53.1 * Revert "(feat) Allow using include to include external YAML files in a config.yaml (BerriAI#6922)" This reverts commit 68e5982. * LiteLLM Minor Fixes & Improvements (11/26/2024) (BerriAI#6913) * docs(config_settings.md): document all router_settings * ci(config.yml): add router_settings doc test to ci/cd * test: debug test on ci/cd * test: debug ci/cd test * test: fix test * fix(team_endpoints.py): skip invalid team object. don't fail `/team/list` call Causes downstream errors if ui just fails to load team list * test(base_llm_unit_tests.py): add 'response_format={"type": "text"}' test to base_llm_unit_tests adds complete coverage for all 'response_format' values to ci/cd * feat(router.py): support wildcard routes in `get_router_model_info()` Addresses BerriAI#6914 * build(model_prices_and_context_window.json): add tpm/rpm limits for all gemini models Allows for ratelimit tracking for gemini models even with wildcard routing enabled Addresses BerriAI#6914 * feat(router.py): add tpm/rpm tracking on success/failure to global_router Addresses BerriAI#6914 * feat(router.py): support wildcard routes on router.get_model_group_usage() * fix(router.py): fix linting error * fix(router.py): implement get_remaining_tokens_and_requests Addresses BerriAI#6914 * fix(router.py): fix linting errors * test: fix test * test: fix tests * docs(config_settings.md): add missing dd env vars to docs * fix(router.py): check if hidden params is dict * LiteLLM Minor Fixes & Improvements (11/27/2024) (BerriAI#6943) * fix(http_parsing_utils.py): remove `ast.literal_eval()` from http utils Security fix - https://huntr.com/bounties/96a32812-213c-4819-ba4e-36143d35e95b?token=bf414bbd77f8b346556e 64ab2dd9301ea44339910877ea50401c76f977e36cdd78272f5fb4ca852a88a7e832828aae1192df98680544ee24aa98f3cf6980d8 bab641a66b7ccbc02c0e7d4ddba2db4dbe7318889dc0098d8db2d639f345f574159814627bb084563bad472e2f990f825bff0878a9 e281e72c88b4bc5884d637d186c0d67c9987c57c3f0caf395aff07b89ad2b7220d1dd7d1b427fd2260b5f01090efce5250f8b56ea2 c0ec19916c24b23825d85ce119911275944c840a1340d69e23ca6a462da610 * fix(converse/transformation.py): support bedrock apac cross region inference Fixes BerriAI#6905 * fix(user_api_key_auth.py): add auth check for websocket endpoint Fixes BerriAI#6926 * fix(user_api_key_auth.py): use `model` from query param * fix: fix linting error * test: run flaky tests first * docs: update the docs (BerriAI#6923) * (bug fix) /key/update was not storing `budget_duration` in the DB (BerriAI#6941) * fix - store budget_duration for keys * test_generate_and_update_key * test_update_user_unit_test * fix user update * (fix) handle json decode errors for DD exception logging (BerriAI#6934) * fix JSONDecodeError * handle async_log_proxy_authentication_errors * fix test_async_log_proxy_authentication_errors_get_request * Revert "Revert "(feat) Allow using include to include external YAML files in a config.yaml (BerriAI#6922)"" This reverts commit 5d13302. * (docs + fix) Add docs on Moderations endpoint, Text Completion (BerriAI#6947) * fix _pass_through_moderation_endpoint_factory * fix route_llm_request * doc moderations api * docs on /moderations * add e2e tests for moderations api * docs moderations api * test_pass_through_moderation_endpoint_factory * docs text completion * (feat) add enforcement for unique key aliases on /key/update and /key/generate (BerriAI#6944) * add enforcement for unique key aliases * fix _enforce_unique_key_alias * fix _enforce_unique_key_alias * fix _enforce_unique_key_alias * test_enforce_unique_key_alias * (fix) tag merging / aggregation logic (BerriAI#6932) * use 1 helper to merge tags + ensure unique ness * test_add_litellm_data_to_request_duplicate_tags * fix _merge_tags * fix proxy utils test * fix doc string * (feat) Allow disabling ErrorLogs written to the DB (BerriAI#6940) * fix - allow disabling logging error logs * docs on disabling error logs * doc string for _PROXY_failure_handler * test_disable_error_logs * rename file * fix rename file * increase test coverage for test_enable_error_logs * fix(key_management_endpoints.py): support 'tags' param on `/key/update` (BerriAI#6945) * LiteLLM Minor Fixes & Improvements (11/29/2024) (BerriAI#6965) * fix(factory.py): ensure tool call converts image url Fixes BerriAI#6953 * fix(transformation.py): support mp4 + pdf url's for vertex ai Fixes BerriAI#6936 * fix(http_handler.py): mask gemini api key in error logs Fixes BerriAI#6963 * docs(prometheus.md): update prometheus FAQs * feat(auth_checks.py): ensure specific model access > wildcard model access if wildcard model is in access group, but specific model is not - deny access * fix(auth_checks.py): handle auth checks for team based model access groups handles scenario where model access group used for wildcard models * fix(internal_user_endpoints.py): support adding guardrails on `/user/update` Fixes BerriAI#6942 * fix(key_management_endpoints.py): fix prepare_metadata_fields helper * fix: fix tests * build(requirements.txt): bump openai dep version fixes proxies argument * test: fix tests * fix(http_handler.py): fix error message masking * fix(bedrock_guardrails.py): pass in prepped data * test: fix test * test: fix nvidia nim test * fix(http_handler.py): return original response headers * fix: revert maskedhttpstatuserror * test: update tests * test: cleanup test * fix(key_management_endpoints.py): fix metadata field update logic * fix(key_management_endpoints.py): maintain initial order of guardrails in key update * fix(key_management_endpoints.py): handle prepare metadata * fix: fix linting errors * fix: fix linting errors * fix: fix linting errors * fix: fix key management errors * fix(key_management_endpoints.py): update metadata * test: update test * refactor: add more debug statements * test: skip flaky test * test: fix test * fix: fix test * fix: fix update metadata logic * fix: fix test * ci(config.yml): change db url for e2e ui testing * bump: version 1.53.1 → 1.53.2 * Updated config.yml --------- Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: paul-gauthier <69695708+paul-gauthier@users.noreply.github.com> Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com> Co-authored-by: Sara Han <127759186+sdiazlor@users.noreply.github.com> * fix(exceptions.py): ensure ratelimit error code == 429, type == "throttling_error" Fixes BerriAI#6973 * fix(utils.py): add jina ai dimensions embedding param support Fixes BerriAI#6591 * fix(exception_mapping_utils.py): add bedrock 'prompt is too long' exception to context window exceeded error exception mapping Fixes BerriAI#6629 Closes BerriAI#6975 * feat(databricks/chat): support structured outputs on databricks Closes BerriAI#6978 - handles content as list for dbrx, - handles streaming+response_format for dbrx * test: fix test * refactor: replace dbrx with 'openai_like' * test: add retry on flaky test * fix(main.py): fix vertex meta llama api call * docs(json_mode.md): update json docs * Update team_endpoints.py (BerriAI#6983) correct the url what should be member_delete and not update * fix: trigger new build * Litellm dev 11 30 2024 (BerriAI#6974) * feat(cohere/chat.py): return citations in model response Closes BerriAI#6814 * fix(cohere/chat.py): fix linting errors * fix(langsmith.py): support 'run_id' for langsmith Fixes BerriAI#6862 * fix(langsmith.py): fix langsmith quickstart Fixes BerriAI#6861 * fix: suppress linting error * LiteLLM Minor Fixes & Improvements (11/29/2024) (BerriAI#6965) * fix(factory.py): ensure tool call converts image url Fixes BerriAI#6953 * fix(transformation.py): support mp4 + pdf url's for vertex ai Fixes BerriAI#6936 * fix(http_handler.py): mask gemini api key in error logs Fixes BerriAI#6963 * docs(prometheus.md): update prometheus FAQs * feat(auth_checks.py): ensure specific model access > wildcard model access if wildcard model is in access group, but specific model is not - deny access * fix(auth_checks.py): handle auth checks for team based model access groups handles scenario where model access group used for wildcard models * fix(internal_user_endpoints.py): support adding guardrails on `/user/update` Fixes BerriAI#6942 * fix(key_management_endpoints.py): fix prepare_metadata_fields helper * fix: fix tests * build(requirements.txt): bump openai dep version fixes proxies argument * test: fix tests * fix(http_handler.py): fix error message masking * fix(bedrock_guardrails.py): pass in prepped data * test: fix test * test: fix nvidia nim test * fix(http_handler.py): return original response headers * fix: revert maskedhttpstatuserror * test: update tests * test: cleanup test * fix(key_management_endpoints.py): fix metadata field update logic * fix(key_management_endpoints.py): maintain initial order of guardrails in key update * fix(key_management_endpoints.py): handle prepare metadata * fix: fix linting errors * fix: fix linting errors * fix: fix linting errors * fix: fix key management errors * fix(key_management_endpoints.py): update metadata * test: update test * refactor: add more debug statements * test: skip flaky test * test: fix test * fix: fix test * fix: fix update metadata logic * fix: fix test * ci(config.yml): change db url for e2e ui testing * test: add more debug logs to langsmith * fix: test change * build(config.yml): fix db url ' * Litellm 12 02 2024 (BerriAI#6994) * add the logprobs param for fireworks ai (BerriAI#6915) * add the logprobs param for fireworks ai * (feat) pass through llm endpoints - add `PATCH` support (vertex context caching requires for update ops) (BerriAI#6924) * add PATCH for pass through endpoints * test_pass_through_routes_support_all_methods * sonnet supports pdf, haiku does not (BerriAI#6928) * (feat) DataDog Logger - Add Failure logging + use Standard Logging payload (BerriAI#6929) * add async_log_failure_event for dd * use standard logging payload for DD logging * use standard logging payload for DD * fix use SLP status * allow opting into _create_v0_logging_payload * add unit tests for DD logging payload * fix dd logging tests * (feat) log proxy auth errors on datadog (BerriAI#6931) * add new dd type for auth errors * add async_log_proxy_authentication_errors * fix comment * use async_log_proxy_authentication_errors * test_datadog_post_call_failure_hook * test_async_log_proxy_authentication_errors * (feat) Allow using include to include external YAML files in a config.yaml (BerriAI#6922) * add helper to process inlcudes directive on yaml * add doc on config management * unit tests for `include` on config.yaml * bump: version 1.52.16 → 1.53. * (feat) dd logger - set tags according to the values set by those env vars (BerriAI#6933) * dd logger, inherit from .envs * test_datadog_payload_environment_variables * fix _get_datadog_service * build(ui/): update ui build * bump: version 1.53.0 → 1.53.1 * Revert "(feat) Allow using include to include external YAML files in a config.yaml (BerriAI#6922)" This reverts commit 68e5982. * LiteLLM Minor Fixes & Improvements (11/26/2024) (BerriAI#6913) * docs(config_settings.md): document all router_settings * ci(config.yml): add router_settings doc test to ci/cd * test: debug test on ci/cd * test: debug ci/cd test * test: fix test * fix(team_endpoints.py): skip invalid team object. don't fail `/team/list` call Causes downstream errors if ui just fails to load team list * test(base_llm_unit_tests.py): add 'response_format={"type": "text"}' test to base_llm_unit_tests adds complete coverage for all 'response_format' values to ci/cd * feat(router.py): support wildcard routes in `get_router_model_info()` Addresses BerriAI#6914 * build(model_prices_and_context_window.json): add tpm/rpm limits for all gemini models Allows for ratelimit tracking for gemini models even with wildcard routing enabled Addresses BerriAI#6914 * feat(router.py): add tpm/rpm tracking on success/failure to global_router Addresses BerriAI#6914 * feat(router.py): support wildcard routes on router.get_model_group_usage() * fix(router.py): fix linting error * fix(router.py): implement get_remaining_tokens_and_requests Addresses BerriAI#6914 * fix(router.py): fix linting errors * test: fix test * test: fix tests * docs(config_settings.md): add missing dd env vars to docs * fix(router.py): check if hidden params is dict * LiteLLM Minor Fixes & Improvements (11/27/2024) (BerriAI#6943) * fix(http_parsing_utils.py): remove `ast.literal_eval()` from http utils Security fix - https://huntr.com/bounties/96a32812-213c-4819-ba4e-36143d35e95b?token=bf414bbd77f8b346556e 64ab2dd9301ea44339910877ea50401c76f977e36cdd78272f5fb4ca852a88a7e832828aae1192df98680544ee24aa98f3cf6980d8 bab641a66b7ccbc02c0e7d4ddba2db4dbe7318889dc0098d8db2d639f345f574159814627bb084563bad472e2f990f825bff0878a9 e281e72c88b4bc5884d637d186c0d67c9987c57c3f0caf395aff07b89ad2b7220d1dd7d1b427fd2260b5f01090efce5250f8b56ea2 c0ec19916c24b23825d85ce119911275944c840a1340d69e23ca6a462da610 * fix(converse/transformation.py): support bedrock apac cross region inference Fixes BerriAI#6905 * fix(user_api_key_auth.py): add auth check for websocket endpoint Fixes BerriAI#6926 * fix(user_api_key_auth.py): use `model` from query param * fix: fix linting error * test: run flaky tests first * docs: update the docs (BerriAI#6923) * (bug fix) /key/update was not storing `budget_duration` in the DB (BerriAI#6941) * fix - store budget_duration for keys * test_generate_and_update_key * test_update_user_unit_test * fix user update * (fix) handle json decode errors for DD exception logging (BerriAI#6934) * fix JSONDecodeError * handle async_log_proxy_authentication_errors * fix test_async_log_proxy_authentication_errors_get_request * Revert "Revert "(feat) Allow using include to include external YAML files in a config.yaml (BerriAI#6922)"" This reverts commit 5d13302. * (docs + fix) Add docs on Moderations endpoint, Text Completion (BerriAI#6947) * fix _pass_through_moderation_endpoint_factory * fix route_llm_request * doc moderations api * docs on /moderations * add e2e tests for moderations api * docs moderations api * test_pass_through_moderation_endpoint_factory * docs text completion * (feat) add enforcement for unique key aliases on /key/update and /key/generate (BerriAI#6944) * add enforcement for unique key aliases * fix _enforce_unique_key_alias * fix _enforce_unique_key_alias * fix _enforce_unique_key_alias * test_enforce_unique_key_alias * (fix) tag merging / aggregation logic (BerriAI#6932) * use 1 helper to merge tags + ensure unique ness * test_add_litellm_data_to_request_duplicate_tags * fix _merge_tags * fix proxy utils test * fix doc string * (feat) Allow disabling ErrorLogs written to the DB (BerriAI#6940) * fix - allow disabling logging error logs * docs on disabling error logs * doc string for _PROXY_failure_handler * test_disable_error_logs * rename file * fix rename file * increase test coverage for test_enable_error_logs * fix(key_management_endpoints.py): support 'tags' param on `/key/update` (BerriAI#6945) * LiteLLM Minor Fixes & Improvements (11/29/2024) (BerriAI#6965) * fix(factory.py): ensure tool call converts image url Fixes BerriAI#6953 * fix(transformation.py): support mp4 + pdf url's for vertex ai Fixes BerriAI#6936 * fix(http_handler.py): mask gemini api key in error logs Fixes BerriAI#6963 * docs(prometheus.md): update prometheus FAQs * feat(auth_checks.py): ensure specific model access > wildcard model access if wildcard model is in access group, but specific model is not - deny access * fix(auth_checks.py): handle auth checks for team based model access groups handles scenario where model access group used for wildcard models * fix(internal_user_endpoints.py): support adding guardrails on `/user/update` Fixes BerriAI#6942 * fix(key_management_endpoints.py): fix prepare_metadata_fields helper * fix: fix tests * build(requirements.txt): bump openai dep version fixes proxies argument * test: fix tests * fix(http_handler.py): fix error message masking * fix(bedrock_guardrails.py): pass in prepped data * test: fix test * test: fix nvidia nim test * fix(http_handler.py): return original response headers * fix: revert maskedhttpstatuserror * test: update tests * test: cleanup test * fix(key_management_endpoints.py): fix metadata field update logic * fix(key_management_endpoints.py): maintain initial order of guardrails in key update * fix(key_management_endpoints.py): handle prepare metadata * fix: fix linting errors * fix: fix linting errors * fix: fix linting errors * fix: fix key management errors * fix(key_management_endpoints.py): update metadata * test: update test * refactor: add more debug statements * test: skip flaky test * test: fix test * fix: fix test * fix: fix update metadata logic * fix: fix test * ci(config.yml): change db url for e2e ui testing * bump: version 1.53.1 → 1.53.2 * Updated config.yml --------- Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: paul-gauthier <69695708+paul-gauthier@users.noreply.github.com> Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com> Co-authored-by: Sara Han <127759186+sdiazlor@users.noreply.github.com> * fix(exceptions.py): ensure ratelimit error code == 429, type == "throttling_error" Fixes BerriAI#6973 * fix(utils.py): add jina ai dimensions embedding param support Fixes BerriAI#6591 * fix(exception_mapping_utils.py): add bedrock 'prompt is too long' exception to context window exceeded error exception mapping Fixes BerriAI#6629 Closes BerriAI#6975 * fix(litellm_logging.py): strip trailing slash for api base Closes BerriAI#6859 * test: skip timeout issue --------- Co-authored-by: ershang-dou <erlie.shang@gmail.com> Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: paul-gauthier <69695708+paul-gauthier@users.noreply.github.com> Co-authored-by: Sara Han <127759186+sdiazlor@users.noreply.github.com> * Litellm test ci cd (BerriAI#6997) * ci(config.yml): run parallel tests * fix(conftest.py): add pytest retry summary * ci(config.yml;): add pip install "pytest-xdist==3.6.1" * ci: revert conftest * (fix) logging Auth errors on datadog (BerriAI#6995) * fix get_standard_logging_object_payload * fix async_post_call_failure_hook * fix post_call_failure_hook * fix change * fix _is_proxy_only_error * fix async_post_call_failure_hook * fix getting request body * remove redundant code * use a well named original function name for auth errors * fix logging auth fails on DD * fix using request body * use helper for _handle_logging_proxy_only_error * (fixes) datadog logging - handle 1MB max log size on DD (BerriAI#6996) * fix dd truncate_standard_logging_payload_content * dd truncate_standard_logging_payload_content * fix test_datadog_payload_content_truncation * add clear msg on _truncate_text * test_truncate_standard_logging_payload * fix linting error * fix linting errors * bump: version 1.53.2 → 1.53.3 --------- Co-authored-by: ershang-dou <erlie.shang@gmail.com> Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: paul-gauthier <69695708+paul-gauthier@users.noreply.github.com> Co-authored-by: Sara Han <127759186+sdiazlor@users.noreply.github.com> Co-authored-by: superpoussin22 <vincent.nadal@orange.fr>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Title
Relevant issues
Closes #6814
Fixes #6862
Type
🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test
Changes
[REQUIRED] Testing - Attach a screenshot of any new tests passing locall
If UI changes, send a screenshot/GIF of working UI fixes