.Net: Batch OpenAI embeddings request

[anthonypuppo]

Motivation and Context

The OpenAI embeddings endpoint and text-embedding-ada-002 supports either a single item or an array of items. Rather than send text inputs one by one, the request should send them all at once for improved performance.

From MS docs:

OpenAI currently allows a larger number of array inputs with text-embedding-ada-002. Azure OpenAI currently supports input arrays up to 16 for text-embedding-ada-002 (Version 2). Both require the max input token limit per API request to remain under 8191 for this model.

Fixes #3294.

Description

Text data is sent as an array to the embeddings endpoint instead of an individual request per text. This also aligns embedding request functionality with how the HuggingFace connector works.

Contribution Checklist

• The code builds clean without any errors or warnings
• The PR follows the SK Contribution Guidelines and the pre-submission formatting script raises no violations
• All unit tests pass, and I have added new tests where possible
• I didn't break anyone 😄

[anthonypuppo]

Non-breaking at compile time but may break at runtime for users that are passing more data then the model allows (combined 16 inputs and 8191 tokens for text-embedding-ada-002).

[dluc]

The current approach allows to pass any number of strings, given that they are processed one at a time. The change requires to limit the list to the max allowed by OpenAI and Azure. It feels like a breaking change. For instance, clients passing hundreds of strings will have to refactor their code taking into account the limits of the model selected.

[markwallace-microsoft]

@anthonypuppo Thanks for your contribution. Based on the commets above it sounds like we don't need this change. I'm going to close this PR but if I'm mistaken please fix the merge conflict and re-open the PR.

[anthonypuppo]

@markwallace-microsoft This change is absolutely necessary.

The current implementation executes an embedding request per input. That's not necessarily wrong, but extremely inefficient as both OpenAI and Azure OpenAI support batching. It's "breaking" due to model limits (i.e. text-embedding-ada-002 only supports up to a max of 16 inputs and ~8k tokens per batch). Existing user implementations could be users passing a large amount of data so they would have to refactor to handle the input chunking according to their user case.

FWIW the Hugging Face connector already does this (no loop, send all inputs at once):

semantic-kernel/dotnet/src/Connectors/Connectors.HuggingFace/TextEmbedding/HuggingFaceTextEmbeddingGenerationService.cs

Lines 105 to 112 in 0772d01

	private async Task<IList<ReadOnlyMemory<float>>> ExecuteEmbeddingRequestAsync(IList<string> data, CancellationToken cancellationToken)
	{
	var embeddingRequest = new TextEmbeddingRequest
	{
	Input = data
	};
	using var httpRequestMessage = HttpRequest.CreatePostRequest(this.GetRequestUri(), embeddingRequest);

Apologies for letting this branch get so out of sync. I'll have it fixed up shortly.