Address M.E.VectorData feedback for IEmbeddingGenerator

[stephentoub]

Necessary, according to @roji:

• Adds a non-generic IEmbeddingGenerator, with GetService then moving down to the non-generic from the generic. This also simplifies the GetService/GetRequiredService extension methods, which no longer need to be generic on TInput/TEmbedding.

Optional (we should discuss which of these we actually want to take):

• Deletes GeneratedEmbeddings and makes GenerateAsync return an IAsyncEnumerable<TEmbedding> instead of GeneratedEmbeddings<TEmbedding>. The positive aspect of this is it allows for embeddings to be streamed as they're generated (though the main embedding generation services today don't stream), and also enables TEmbedding to be covariant (though it's not clear how much benefit that actually yields). The negative is we lose a place to hang metadata that's about the whole batch, in particular usage data. This moves that usage data to instead be on the Embedding, which is helpful for cases where such information is available per embedding, but isn't ideal for cases where it's about the whole batch. This PR deals with that by just putting such information onto one of the generated embeddings. We'll need to discuss the tradeoff. The alternative is we keep GeneratedEmbeddings but make it an IAsyncEnumerable instead of an IList. That gives a place to hang such batch information, but only if it's all available at the start of the streaming; however, it's also a bit harder to consume, as you need to await the initial call and then await foreach the embeddings.
• Adds a SupportedInputMediaTypes set to EmbeddingGeneratorMetadata, so that a generator can be explicit about what inputs it supports.
• Adds an optional MediaType parameter to TextContent. This would allow, for example, tagging code by language.
• Makes the DataContent's ReadOnlyMemory constructor require a media type to be supplied. Ideally we'd also require this for non-data urls, but that's a bit trickier, as we can't distinguish between data and non-data urls in the type system. We could check for it dynamically. (This one wasn't actually mentioned by @roji.)

Not addressed:

• Removing IDisposable from IEmbeddingGenerator (and IChatClient), or adding IAsyncDisposable. Not having these implementations makes middleware awkward, as all middleware components need to expose the interface. It also makes it harder to know you might need to dispose the result of a method like AsEmbeddingGenerator. To my knowledge, we don't know of cases where async disposal is necessary.
• Add back a Metadata property. I still think GetService<EmbeddingGeneratorMetadata>() is sufficient, but we can discuss it further. We might want to consider an analyzer that would make diagnostic suggestions about supporting it in GetService and filling in as much data as possible.
• Splitting off a UrlContent from DataContent, to differentiate in a strongly typed way cases where data is included or not.

cc: @roji, @SteveSandersonMS

Microsoft Reviewers: Open in CodeFlow

[roji]

Adds an optional MediaType parameter to TextContent. This would allow, for example, tagging code by language.

Specifically for language, IIRC MIME types don't officially support that - maybe text/plan;lang=en-US is OK, but I don't think it's standard (that's why HTTP has a separate Content-Language header?). If we wanted to, we could add a Language property to TextContent, though before that we'd probably want to see evidence of actual embedding generation services that accept a language parameter (I'd expect them to maybe auto-detect?) - @westey-m did you encounter any that do?

[stephentoub]

Specifically for language, IIRC MIME types don't officially support that

I was referring to programming languages, for eg code generated by the llm as part of a code interpreter tool, like text/x-python, text/x-csharp, text/html, or application/typescript.

[stephentoub]

@roji, @westey-m, @SteveSandersonMS, I've pushed a revision based on our discussion. This now only includes:

• Lowering GetService down to a non-generic IEmbeddingGenerator
• Splitting DataContent into DataContent and UriContent. The former requires bytes, the latter is about remote content.

@rogerbarreto, FYI

@shyamnamboodiripad, @peterwald, I don't think this should affect you much if at all, but FYI.

[stephentoub]

cc: @luisquintanilla, since you were previously asking about DataContent representing both in-memory data and remote content

[SteveSandersonMS]

Nice!

[roji]

LGTM (see comments)!

[stephentoub]

Addressed feedback:

• Renamed MediaTypeStartsWith to HasTopLevelMediaType and changed it to compare just the top-level type portion of the media type, e.g. content.HasTopLevelMediaType("image").
• Changed Add{Keyed}EmbeddingGenerator to register for both the generic and non-generic interfaces.

[rogerbarreto]

Once the _dataUri is generated from the _data, since it's a readonly whouldn't it be nice to cache it and vice-versa?

Suggested change

	_uri = string.Concat("data:", MediaType, ";base64,", Convert.ToBase64String(_data.GetValueOrDefault()
	_uri = string.Concat("data:", MediaType, ";base64,", Convert.ToBase64String(_data.GetValueOrDefault()

[stephentoub]

Where would _dataUri end up being used again?