/v1/inference/embeddings input and output shape mismatch

[mattf]

/v1/inference/embeddings: model x List[InterleavedContent] -> List[List[float]]

the shape mismatch comes from InterleavedContent allowing for List[InterleavedContentItem].

example: [string, [text0, text1], image] -?-> [embedding of string, embedding of text0, embedding of text1, embedding of image]

i suggest aligning the shapes.

my preference is to change the input shape, and use an input of array of string | array of InterleavedContentItem, which keeps string (untyped) and text / image (typed) inputs separate.

a further enhancement: embedding is often done in two modes, batch and query. in batch mode many items are embedded for storage. in query mode a single item is embedded for lookup. allowing input of string | array of string | array of InterleavedContentItem facilitates this use case.

[mattf]

cc @raghotham @ashwinb @yanxi0830

[ashwinb]

Good spot. I definitely agree with at least changing it to List[InterleavedContentItem] immediately otherwise the contract is broken.