Skip to content

Commit

Permalink
Renamed the function to disable the env var
Browse files Browse the repository at this point in the history
  • Loading branch information
ivanleomk committed Jun 13, 2024
1 parent 955d247 commit 0b3e28e
Show file tree
Hide file tree
Showing 4 changed files with 48 additions and 9 deletions.
39 changes: 39 additions & 0 deletions docs/concepts/reask_validation.md
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,45 @@ The docs are currently incomplete, but we have a few advanced validation techniq
1. Validate the entire object with all attributes rather than one attribute at a time
2. Using some 'context' to validate the object: In this case, we use the `context` to check if the citation existed in the original text.

## Optimizing Token usage

Pydantic automatically includes a URL within the error message itself when an error is thrown so that users can learn more about the specific error that was thrown. Some users might want to remove this URL since it adds extra tokens that otherwise might not add much value to the validation process.

We've created a small helper function that you can use below which removes this url in the error message

```python hl_lines="6"
from instructor.utils import disable_pydantic_error_url
from pydantic import BaseModel, ValidationError
from typing_extensions import Annotated
from pydantic import AfterValidator

disable_pydantic_error_url() # (1)!


def name_must_contain_space(v: str) -> str:
if " " not in v:
raise ValueError("Name must contain a space.")
return v.lower()


class UserDetail(BaseModel):
age: int
name: Annotated[str, AfterValidator(name_must_contain_space)]


try:
person = UserDetail(age=29, name="Jason")
except ValidationError as e:
print(e)
"""
1 validation error for UserDetail
name
Value error, Name must contain a space. [type=value_error, input_value='Jason', input_type=str]
"""
```

1. We disable the error by setting an environment variable `PYDANTIC_ERRORS_INCLUDE_URL` to `0`. This is valid only for the duration that the script is executed for, once the function is not called, the original behaviour is restored.

## Takeaways

By integrating these advanced validation techniques, we not only improve the quality and reliability of LLM-generated content, but also pave the way for more autonomous and effective systems.
2 changes: 0 additions & 2 deletions instructor/__init__.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
import importlib.util

from .mode import Mode
from .utils import set_env_variable
from .process_response import handle_response_model
from .distil import FinetuneFormat, Instructions
from .dsl import (
Expand All @@ -23,7 +22,6 @@
Provider,
)

set_env_variable()

__all__ = [
"Instructor",
Expand Down
8 changes: 4 additions & 4 deletions instructor/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -230,6 +230,10 @@ def __get__(self, instance: object, cls: type[Any]) -> R_co:
return self.cproperty(cls)


def disable_pydantic_error_url():
os.environ["PYDANTIC_ERRORS_INCLUDE_URL"] = "0"


def transform_to_gemini_prompt(
messages_chatgpt: list[ChatCompletionMessageParam],
) -> list[dict[str, Any]]:
Expand All @@ -250,7 +254,3 @@ def transform_to_gemini_prompt(
messages_gemini[0]["parts"].insert(0, f"*{system_prompt}*")

return messages_gemini


def set_env_variable():
os.environ["PYDANTIC_ERRORS_INCLUDE_URL"] = "0"
8 changes: 5 additions & 3 deletions tests/test_function_calls.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
import instructor
from instructor import OpenAISchema, openai_schema
from instructor.exceptions import IncompleteOutputException
from instructor.utils import disable_pydantic_error_url

T = TypeVar("T")

Expand Down Expand Up @@ -190,12 +191,13 @@ def test_control_characters_allowed_in_anthropic_json_non_strict_mode(

def test_pylance_url_config() -> None:
class Model(BaseModel):
list_of_ints: List[int] = None
a_float: float = None
list_of_ints: list[int]
a_float: float

disable_pydantic_error_url()
data = dict(list_of_ints=["1", 2, "bad"], a_float="Not a float")

try:
Model(**data)
Model(**data) # type: ignore
except ValidationError as e:
assert "https://errors.pydantic.dev" not in str(e)

0 comments on commit 0b3e28e

Please sign in to comment.