Merge branch 'main' into audio

docs/cli/batch.md

@@ -3,23 +3,32 @@ title: Managing Batch Jobs with OpenAI CLI
 description: Learn how to create, list, and cancel batch jobs using the OpenAI Command Line Interface (CLI) for efficient job management.
 ---
 
-# Using the Command Line Interface
+# Using the Command Line Interface for Batch Jobs
 
-The instructor CLI provides functionalities for managing batch jobs on OpenAI
+
+The instructor CLI provides functionalities for managing batch jobs on both OpenAI and Anthropic platforms. This dual support allows users to leverage the strengths of both providers for their batch processing needs.
+
+## Supported Providers
+
+- **OpenAI**: Utilizes OpenAI's robust batch processing capabilities.
+- **Anthropic**: Leverages Anthropic's advanced language models for batch operations.
+
+To switch between providers, use the `--use-anthropic` flag in the relevant commands.
 
 ```bash
 $ instructor batch --help
 
  Usage: instructor batch [OPTIONS] COMMAND [ARGS]...
 
- Manage OpenAI Batch jobs
+ Manage OpenAI and Anthropic Batch jobs
 
 ╭─ Options ───────────────────────────────────────────────────────────────────────────╮
 │ --help Show this message and exit. │
 ╰─────────────────────────────────────────────────────────────────────────────────────╯
 ╭─ Commands ──────────────────────────────────────────────────────────────────────────╮
 │ cancel Cancel a batch job │
 │ create-from-file Create a batch job from a file │
+│ download-file Download the file associated with a batch job │
 │ list See all existing batch jobs │
 ╰─────────────────────────────────────────────────────────────────────────────────────╯
 ```
@@ -43,14 +52,16 @@ $ instructor batch list --help
 │ [default: 10] │
 │ --screen --no-screen Enable or disable screen output │
 │ [default: no-screen] │
+│ --use-anthropic Use Anthropic API instead of OpenAI │
+│ [default: False] │
 │ --help Show this message and exit. │
 ╰─────────────────────────────────────────────────────────────────────────────────────╯
 ```
 
-This returns a list of jobs as seen below
+This returns a list of jobs as seen below:
 
-```
-$ instructor batch list --limit 9
+```bash
+$ instructor batch list --limit 5
 
  OpenAI Batch Jobs
 ┏━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━┓
@@ -61,58 +72,50 @@ $ instructor batch list --limit 9
 │ batch_zsTSsWVLgpEan… │ 2024-06-19 15:06:05 │ completed │ 0 │ 15 │ 15 │
 │ batch_igaa2j9VBVw2Z… │ 2024-06-19 15:01:59 │ completed │ 0 │ 300 │ 300 │
 │ batch_HcjI2wG46Y1LY… │ 2024-06-12 15:45:37 │ completed │ 0 │ 3 │ 3 │
-│ batch_YiRKLAmKBhwxM… │ 2024-06-12 15:09:44 │ completed │ 0 │ 3 │ 3 │
-│ batch_hS0XGlXzTVS7S… │ 2024-06-12 15:05:59 │ completed │ 0 │ 3 │ 3 │
-│ batch_6s4FmcaV7woam… │ 2024-06-12 14:26:34 │ completed │ 0 │ 3 │ 3 │
 └──────────────────────┴─────────────────────┴───────────┴────────┴───────────┴───────┘
 ```
 
 ### Create From File
 
-You'll need to supply a valid .jsonl file in order to be able to create a Batch job.
-
-??? info "Don't have a `.jsonl` file on hand?"
-
- You can use Instructor to create the `.jsonl` with nothing more than simple pydantic and our `BatchJob` object as seen below.
-
- ```python
- from instructor.batch import BatchJob
- from pydantic import BaseModel, Field
- from typing import Literal
-
+You'll need to supply a valid .jsonl file to create a Batch job. Here's how you can create one using Instructor:
 
- class Classification(BaseModel):
-  label: Literal["SPAM", "NOT_SPAM"] = Field(
-  ..., description="Whether the email is spam or not"
-  )
+```python
+from instructor.batch import BatchJob
+from pydantic import BaseModel, Field
+from typing import Literal
 
+class Classification(BaseModel):
+ label: Literal["SPAM", "NOT_SPAM"] = Field(
+ ..., description="Whether the email is spam or not"
+ )
 
- emails = [
-  "Hello there I'm a Nigerian prince and I want to give you money",
-  "Meeting with Thomas has been set at Friday next week",
-  "Here are some weekly product updates from our marketing team",
+emails = [
+ "Hello there I'm a Nigerian prince and I want to give you money",
+ "Meeting with Thomas has been set at Friday next week",
+ "Here are some weekly product updates from our marketing team",
+]
+
+messages = [
+ [
+ {
+ "role": "system",
+ "content": f"Classify the following email {email}",
+ }
  ]
+ for email in emails
+]
 
- messages = [
-  [
-  {
-  "role": "user",
-  "content": f"Classify the following email {email}",
-  }
-  ]
-  for email in emails
- ]
+import json
 
- BatchJob.create_from_messages(
+with open("output.jsonl", "w") as f:
+ for line in BatchJob.create_from_messages(
  messages,
  model="gpt-3.5-turbo",
  response_model=Classification,
  max_tokens=100,
-  file_path="output.jsonl"
- )
- ```
-
-You can then import in the .jsonl file using the `instructor batch create-from-file` command
+ ):
+ f.write(json.dumps(line) + "\n")
+```
 
 ```bash
 $ instructor batch create-from-file --help
@@ -124,13 +127,21 @@ Usage: instructor batch create-from-file [OPTIONS]
 ╭─ Options ───────────────────────────────────────────────────────────────────────────╮
 │ * --file-path TEXT File containing the batch job requests [default: None] │
 │ [required] │
+│ --use-anthropic Use Anthropic API instead of OpenAI │
+│ [default: False] │
 │ --help Show this message and exit. │
 ╰─────────────────────────────────────────────────────────────────────────────────────╯
 ```
 
+Example usage:
+
+```bash
+$ instructor batch create-from-file --file-path output.jsonl
+```
+
 ### Cancelling a Batch Job
 
-You can also cancel an outstanding batch job by using the `cancel` command.
+You can cancel an outstanding batch job using the `cancel` command:
 
 ```bash
 $ instructor batch cancel --help
@@ -141,6 +152,42 @@ $ instructor batch cancel --help
 
 ╭─ Options ───────────────────────────────────────────────────────────────────────────╮
 │ * --batch-id TEXT Batch job ID to cancel [default: None] [required] │
-│ --help Show this message and exit. │
+│ --use-anthropic Use Anthropic API instead of OpenAI │
+│ [default: False] │
+│ --help Show this message and exit. │
 ╰─────────────────────────────────────────────────────────────────────────────────────╯
 ```
+
+Example usage:
+
+```bash
+$ instructor batch cancel --batch-id batch_BSMSiMMy8on2D
+```
+
+### Downloading Batch Job Results
+
+To download the results of a completed batch job:
+
+```bash
+$ instructor batch download-file --help
+
+ Usage: instructor batch download-file [OPTIONS]
+
+ Download the file associated with a batch job
+
+╭─ Options ───────────────────────────────────────────────────────────────────────────╮
+│ * --batch-id TEXT Batch job ID to download [default: None] [required] │
+│ * --download-file-path TEXT Path to download file to [default: None] [required] │
+│ --use-anthropic Use Anthropic API instead of OpenAI │
+│ [default: False] │
+│ --help Show this message and exit. │
+╰─────────────────────────────────────────────────────────────────────────────────────╯
+```
+
+Example usage:
+
+```bash
+$ instructor batch download-file --batch-id batch_pD5dqHmqjWYF5 --download-file-path results.jsonl
+```
+
+This comprehensive set of commands allows you to manage batch jobs efficiently, whether you're using OpenAI or Anthropic as your provider.

docs/concepts/multimodal.md

@@ -1,6 +1,6 @@
 ---
 title: Seamless Multimodal Interactions with Instructor
-description: Learn how the Image class in Instructor enables seamless handling of images and text across different AI models.
+description: Learn how the Image and Audio class in Instructor enables seamless handling of images, audio and text across different AI models.
 ---
 
 # Multimodal
@@ -37,6 +37,57 @@ response = client.chat.completions.create(
 
 The `Image` class takes care of the necessary conversions and formatting, ensuring that your code remains clean and provider-agnostic. This flexibility is particularly valuable when you're experimenting with different models or when you need to switch providers based on specific project requirements.
 
+By leveraging Instructor's multimodal capabilities, you can focus on building your application logic without worrying about the intricacies of each provider's image handling format. This not only saves development time but also makes your code more maintainable and adaptable to future changes in AI provider APIs.
+
+Alternatively, by passing `autodetect_images=True` to `client.chat.completions.create`, you can pass file paths, URLs, or base64 encoded content directly as strings.
+
+```python
+import instructor
+import openai
+
+client = instructor.from_openai(openai.OpenAI())
+
+response = client.chat.completions.create(
+ model="gpt-4o-mini",
+ response_model=ImageAnalyzer,
+ messages=[
+ {"role": "user", "content": ["What is in this two images?", "https://example.com/image.jpg", "path/to/image.jpg"]}
+ ],
+ autodetect_images=True
+)
+```
+
+### Anthropic Prompt Caching
+Instructor supports Anthropic prompt caching with images. To activate prompt caching, you can pass image content as a dictionary of the form
+```python
+{"type": "image", "source": <path_or_url_or_base64_encoding>, "cache_control": True}
+```
+and set `autodetect_images=True`, or flag it within a constructor such as `instructor.Image.from_path("path/to/image.jpg", cache_control=True)`. For example:
+
+```python
+import instructor
+from anthropic import Anthropic
+
+client = instructor.from_anthropic(Anthropic(), enable_prompt_caching=True)
+
+cache_control = {"type": "ephemeral"}
+response = client.chat.completions.create(
+ model="claude-3-haiku-20240307",
+ response_model=ImageAnalyzer, # This can be set to `None` to return an Anthropic prompt caching message
+ messages=[
+ {
+ "role": "user",
+ "content": [
+ "What is in this two images?",
+ {"type": "image", "source": "https://example.com/image.jpg", "cache_control": cache_control},
+ {"type": "image", "source": "path/to/image.jpg", "cache_control": cache_control},
+ ]
+ }
+ ],
+ autodetect_images=True
+)
+```
+
 ## `Audio`
 
 The `Audio` class represents an audio file that can be loaded from a URL or file path. It provides methods to create `Audio` instances but currently only OpenAI supports it. You can create an instance using the `from_path` and `from_url` methods. The `Audio` class will automatically convert it to a base64-encoded image and include it in the API request.
@@ -66,10 +117,6 @@ resp = client.chat.completions.create(
  response_model=User,
  modalities=["text"],
  audio={"voice": "alloy", "format": "wav"},
- messages=[
- {
- "role": "user",
- "content": [
  "Extract the following information from the audio:",
  Audio.from_path("./output.wav"),
  ],
@@ -79,4 +126,3 @@ resp = client.chat.completions.create(
 
 print(resp)
 # > name='Jason' age=20
-```