Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(community): AILanguageModel interface #7025

Merged
merged 2 commits into from
Oct 20, 2024
Merged

fix(community): AILanguageModel interface #7025

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f2836b7
fix: AILanguageModel interface
hemanth Oct 18, 2024
9ec87a6
Format and lint
jacoblee93 Oct 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 69 additions & 12 deletions libs/langchain-community/src/experimental/llms/chrome_ai.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,25 +5,82 @@
import { IterableReadableStream } from "@langchain/core/utils/stream";
import { BaseLLMParams, LLM } from "@langchain/core/language_models/llms";

export interface AI {
canCreateTextSession(): Promise<AIModelAvailability>;
createTextSession(options?: AITextSessionOptions): Promise<AITextSession>;
defaultTextSessionOptions(): Promise<AITextSessionOptions>;
export interface AILanguageModelFactory {
create(options?: AILanguageModelCreateOptions): Promise<AILanguageModel>;
capabilities(): Promise<AILanguageModelCapabilities>;
}

export interface AITextSession {
prompt(input: string): Promise<string>;
promptStreaming(input: string): ReadableStream;
export interface AILanguageModel extends EventTarget {
prompt(input: AILanguageModelPromptInput, options?: AILanguageModelPromptOptions): Promise<String>;

Check failure on line 14 in libs/langchain-community/src/experimental/llms/chrome_ai.ts

View workflow job for this annotation

GitHub Actions / Check linting 

Don't use `String` as a type. Use string instead
promptStreaming(input: AILanguageModelPromptInput, options?: AILanguageModelPromptOptions): ReadableStream;

countPromptTokens(input: AILanguageModelPromptInput, options?: AILanguageModelPromptOptions): Promise<number>;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have updated the interface to the latest definition, we may implement these if it make sense.

Copy link
Collaborator

@jacoblee93 jacoblee93 Oct 19, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given this will likely change again, is it worth adding all of this if we're not using it?

Would prefer to keep it minimal, types will likely come from elsewhere eventually

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Eh it's experimental I suppose so it's fine


get maxTokens(): number;
get tokensSoFar(): number;
get tokensLeft(): number;

get topK(): number;
get temperature(): number;

oncontextoverflow: ((event: Event) => void);

clone(options?: AILanguageModelCloneOptions): Promise<AILanguageModel>;
destroy(): void;
clone(): AITextSession;
}

export interface AITextSessionOptions {
interface AILanguageModelCapabilities {
readonly available: AICapabilityAvailability;
languageAvailable(languageTag: string): AICapabilityAvailability;

get defaultTopK(): number | undefined;
get maxTopK(): number | undefined;
get defaultTemperature(): number | undefined;
get maxTemperature(): number | undefined;
}

interface AILanguageModelCreateOptions {
signal?: AbortSignal;
monitor?: AICreateMonitorCallback;
systemPrompt?: string;
initialPrompts?: AILanguageModelInitialPrompt[];
topK: number;
temperature: number;
}

export type AIModelAvailability = "readily" | "after-download" | "no";
export interface AILanguageModelInitialPrompt {
role: AILanguageModelInitialPromptRole;
content: string;
}

export interface AILanguageModelPrompt {
role: AILanguageModelPromptRole;
content: string;
}

export interface AILanguageModelPromptOptions {
signal?: AbortSignal;
}

export interface AILanguageModelCloneOptions {
signal?: AbortSignal;
}

export type AILanguageModelPromptInput = string | AILanguageModelPrompt | AILanguageModelPrompt[];

enum AILanguageModelInitialPromptRole {
"system",
"user",
"assistant"
}

enum AILanguageModelPromptRole {
"user",
"assistant"
}

export type AICapabilityAvailability = "yes" | "no";
export type AICreateMonitorCallback = () => void;

export interface ChromeAIInputs extends BaseLLMParams {
topK?: number;
Expand Down Expand Up @@ -93,14 +150,14 @@
`Could not initialize ChromeAI instance. Make sure you are running a version of Chrome with the proper experimental flags enabled.\n\nError message: ${e.message}`
);
}
const { available } = await aiInstance.assistant.capabilities();
const { available } = await aiInstance.languageModel.capabilities();
if (available === "no") {
throw new Error("The AI model is not available.");
} else if (available === "after-download") {
throw new Error("The AI model is not yet downloaded.");
}

const session = await aiInstance.assistant.create({
const session = await aiInstance.languageModel.create({
systemPrompt: this.systemPrompt,
topK: this.topK,
temperature: this.temperature,
Expand Down
Loading