Skip to content

feat(js): add llama-3.2 support to model garden plugin #969

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Sep 26, 2024
Merged

feat(js): add llama-3.2 support to model garden plugin #969

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
a8656e4
feat(js): Support llama3.2 on model garden
huangjeff5 Sep 24, 2024
0b780c1
fix lint
huangjeff5 Sep 24, 2024
4c7104c
export llama32
huangjeff5 Sep 24, 2024
9497394
wip
huangjeff5 Sep 24, 2024
00a5101
wip
huangjeff5 Sep 24, 2024
d85d98a
Merge branch 'main' into jh-model-garden-llama32
huangjeff5 Sep 24, 2024
282ca02
turn on prod endpoint
huangjeff5 Sep 25, 2024
f276ae3
revert js/testapps/flow-simple-ai/src/index.ts
huangjeff5 Sep 26, 2024
122780c
Update model_garden.ts
huangjeff5 Sep 26, 2024
d292ddb
Update model_garden.ts
huangjeff5 Sep 26, 2024
e115678
Update model_garden.ts
huangjeff5 Sep 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions js/plugins/vertexai/src/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ import {
SUPPORTED_OPENAI_FORMAT_MODELS,
llama3,
llama31,
llama32,
modelGardenOpenaiCompatibleModel,
} from './model_garden.js';
import {
Expand Down Expand Up @@ -104,6 +105,7 @@ export {
imagen3Fast,
llama3,
llama31,
llama32,
textEmbedding004,
textEmbeddingGecko,
textEmbeddingGecko001,
Expand Down
19 changes: 18 additions & 1 deletion js/plugins/vertexai/src/model_garden.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,23 @@ export const ModelGardenModelConfigSchema = OpenAIConfigSchema.extend({
location: z.string().optional(),
});

export const llama32 = modelRef({
name: 'vertexai/llama-3.2',
info: {
label: 'Llama 3.2',
supports: {
multiturn: true,
tools: true,
media: true,
systemRole: true,
output: ['text', 'json'],
},
versions: ['meta/llama-3.2-90b-vision-instruct-maas'],
},
configSchema: ModelGardenModelConfigSchema,
version: 'meta/llama-3.2-90b-vision-instruct-maas',
});

export const llama31 = modelRef({
name: 'vertexai/llama-3.1',
info: {
Expand Down Expand Up @@ -72,6 +89,7 @@ export const llama3 = modelRef({
export const SUPPORTED_OPENAI_FORMAT_MODELS = {
'llama3-405b': llama3,
'llama-3.1': llama31,
'llama-3.2': llama32,
};

export function modelGardenOpenaiCompatibleModel(
Expand All @@ -87,7 +105,6 @@ export function modelGardenOpenaiCompatibleModel(
baseUrlTemplate =
'https://{location}-aiplatform.googleapis.com/v1beta1/projects/{projectId}/locations/{location}/endpoints/openapi';
}

const clientFactory = async (
request: GenerateRequest<typeof ModelGardenModelConfigSchema>
): Promise<OpenAI> => {
Expand Down
Loading