Skip to content

Fix chunking strategy serialization and field visibility #288

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Nov 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 8 additions & 3 deletions async-openai/src/types/assistant.rs
Original file line number Diff line number Diff line change
Expand Up @@ -60,19 +60,24 @@ pub struct AssistantVectorStore {
pub enum AssistantVectorStoreChunkingStrategy {
/// The default strategy. This strategy currently uses a `max_chunk_size_tokens` of `800` and `chunk_overlap_tokens` of `400`.
#[default]
#[serde(rename = "auto")]
Auto,
Static(StaticChunkingStrategy),
#[serde(rename = "static")]
Static {
#[serde(rename = "static")]
config: StaticChunkingStrategy,
},
}

/// Static Chunking Strategy
#[derive(Clone, Serialize, Debug, Deserialize, PartialEq, Default)]
pub struct StaticChunkingStrategy {
/// The maximum number of tokens in each chunk. The default value is `800`. The minimum value is `100` and the maximum value is `4096`.
max_chunk_size_tokens: u16,
pub max_chunk_size_tokens: u16,
/// The number of tokens that overlap between chunks. The default value is `400`.
///
/// Note that the overlap must not exceed half of `max_chunk_size_tokens`.
chunk_overlap_tokens: u16,
pub chunk_overlap_tokens: u16,
}

/// Represents an `assistant` that can call the model and use tools.
Expand Down
7 changes: 6 additions & 1 deletion async-openai/src/types/vector_store.rs
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,13 @@ pub struct CreateVectorStoreRequest {
pub enum VectorStoreChunkingStrategy {
/// The default strategy. This strategy currently uses a `max_chunk_size_tokens` of `800` and `chunk_overlap_tokens` of `400`.
#[default]
#[serde(rename = "auto")]
Auto,
Static(StaticChunkingStrategy),
#[serde(rename = "static")]
Static {
#[serde(rename = "static")]
config: StaticChunkingStrategy,
},
}

/// Vector store expiration policy
Expand Down