Create a GetChunks extension method for batches

[juchom]

Pull Request

What does this PR do?

This pull request is here to make code a little bit easier to follow for batches.

For exemple if we look at AddDocumentsInBatchesAsync we have this two piece of code.

meilisearch-dotnet/src/Meilisearch/Index.cs

Lines 152 to 161 in 3f14f43

	public async Task<IEnumerable<TaskInfo>> AddDocumentsInBatchesAsync<T>(IEnumerable<T> documents, int batchSize = 1000, string primaryKey = default, CancellationToken cancellationToken = default)
	{
	async Task AddActionAsync(List<T> items, List<TaskInfo> tasks, CancellationToken localCancellationToken)
	{
	tasks.Add(await this.AddDocumentsAsync(items, primaryKey, localCancellationToken).ConfigureAwait(false));
	}
	var result = await BatchOperationAsync(documents, batchSize, AddActionAsync).ConfigureAwait(false);
	return result;
	}

meilisearch-dotnet/src/Meilisearch/Index.cs

Lines 732 to 745 in 3f14f43

	private static async Task<List<TaskInfo>> BatchOperationAsync<T>(IEnumerable<T> items, int batchSize, Func<List<T>, List<TaskInfo>, CancellationToken, Task> action, CancellationToken cancellationToken = default)
	{
	var itemsList = new List<T>(items);
	var numberOfBatches = Math.Ceiling((double)itemsList.Count / batchSize);
	var result = new List<TaskInfo>();
	for (var i = 0; i < numberOfBatches; i++)
	{
	var actualSize = Math.Min(batchSize, itemsList.Count - (i * batchSize));
	var batch = itemsList.GetRange(i * batchSize, actualSize);
	await action.Invoke(batch, result, cancellationToken).ConfigureAwait(false);
	}
	return result;
	}

To follow the code we have to start 159 with a call to the second snippet. We go to the second snippet where we do so computation to get chunks, then we call a function and we have to go up to the first snippet and see what is happening with this local function.

With this refactoring the AddDocumentsInBatchesAsync is self explaining.

The second snippet is now replaced by an extension method that returns a chunk on demand.

With the new code you see what the method is doing on first read

meilisearch-dotnet/src/Meilisearch/Index.cs

Lines 152 to 161 in 4264a71

	public async Task<IEnumerable<TaskInfo>> AddDocumentsInBatchesAsync<T>(IEnumerable<T> documents, int batchSize = 1000, string primaryKey = default, CancellationToken cancellationToken = default)
	{
	var tasks = new List<TaskInfo>();
	foreach (var chunk in documents.GetChunks(batchSize))
	{
	tasks.Add(await this.AddDocumentsAsync(chunk, primaryKey, cancellationToken).ConfigureAwait(false));
	}
	return tasks;
	}

This is a part from a previous PR #213. I will make several from #213 because there is too much things in this one.

[brunoocasali]

Hey @juchom I'll take some time to review your PR, but I didn’t forget you!

Anyway, thanks for your contribution, I'll let you know if I need anything!

[brunoocasali]

Thank you @juchom🥇

Your description really helped me to understand what was going on there, the improvement made a real difference, thanks! 🎉

[brunoocasali]

bors merge

[bors]

Build failed:

• format-check

[brunoocasali]

bors try

[bors]

try

Build failed:

• format-check

[brunoocasali]

bors try

[bors]

try

Build succeeded:

• format-check
• integration-tests

[brunoocasali]

FINALLY! 🔥 🔥

Thanks a lot @juchom

[brunoocasali]

bors merge

[bors]

Build succeeded:

• format-check
• integration-tests