Benchmarking: Adds use of ARM Templates for benchmarking (#3838)

* initial commit DONT REVIEW

* fixes and documentation

* Apply suggestions from code review

Co-authored-by: Matias Quaranta <ealsur@users.noreply.github.com>

* requested changes

* Apply suggestions from code review

Co-authored-by: Matias Quaranta <ealsur@users.noreply.github.com>

* name changes

* readme changes

* nits + changing case of parameters file

---------

Co-authored-by: Matias Quaranta <ealsur@users.noreply.github.com>

Microsoft.Azure.Cosmos.Samples/Tools/Benchmark/ARMTemplate/README.md

@@ -0,0 +1,73 @@
+# Running benchmarks on ARM Tempaltes
+
+[ARM Templates](https://learn.microsoft.com/azure/azure-resource-manager/templates/) makes executing the Azure Cosmos DB SDK Benchmark extremely easy, with very few steps involved. Plus, it lets you test and evaluate performance quickly on multiple resource (CPU/RAM) configurations and across multiple Azure regions seamlessly.
+
+For the below steps, you will **need an Azure Subscription**.
+
+## Steps
+
+### Deploy with one click
+
+Just click in the **Deploy to Azure button** and it will guide you into automatically configuring, deploying, and running the benchmark.
+
+[![Deploy to Azure](https://aka.ms/deploytoazurebutton)](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2FAzure%2Fazure-cosmos-dotnet-v3%2Fmaster%2FARMBenchmarking%2FMicrosoft.Azure.Cosmos.Samples%2FTools%2FBenchmark%2FARMTemplate%2FbenchmarkTemplate.json)
+
+Please populate the `endpoint` and `key` for your Azure Cosmos DB account. You can [obtain these from the Azure Portal or through CLI](https://learn.microsoft.com/azure/cosmos-db/secure-access-to-data?tabs=using-primary-key#primary-keys).
+
+Optionally you can modify the other parameters, such as the `throughput` for the container that will get created, the amount of `documents` to insert, the degree of `parallelism`, and if you want the container to be deleted after the benchmark is run (`cleanUpOnFinish` `true/false`).
+
+Additionally, the template lets you customize the size of the container instance that will be deployed, which you can make as similar as possible to the instance you will be running in production to simulate results.
+
+### Deploy with Azure CLI
+
+First open the `parameters.json` file and populate the `endpoint` and `key` for your Azure Cosmos DB account. You can [obtain these from the Azure Portal or through CLI](https://learn.microsoft.com/azure/cosmos-db/secure-access-to-data?tabs=using-primary-key#primary-keys).
+
+ Next, modify the other parameters, such as the `throughput` for the container that will get created, the amount of `documents` to insert, the degree of `parallelism`, and if you want the container to be deleted after the benchmark is run (`cleanUpOnFinish` `true/false`).
+
+ If you're deploying the template to a resource group that does not exist you must create one first. Please note that the name of the resource group can only include alphanumeric characters, periods, underscores, hyphens, and parenthesis. It can be up to 90 characters. The name can't end in a period. To create a resource group use the following command:
+ ```bash
+ az group create --name $resourceGroupName --location $location
+ ```
+
+ To run the benchmark first navigate to the directory the `benchmarkTemplate.json` and `parameters.json` files are stored and use the Azure CLI with the following command:
+ ```bash
+ az deployment group create --resource-group $resourceGroupName --template-file benchmarkTemplate.json --parameters @parameters.json
+ ```
+
+### The Benchmark 
+
+Once you create the benchmark, it will do the following:
+
+1. It will create a Linux container named `cosmosdbsdkperf` and provision an image with NET 6 inside an instance with the configured CPU and RAM.
+![Provisioned Container Instance](./arm1.png)
+2. Clone the Azure Cosmos DB SDK repository with the required files
+3. Execute the benchmark and provide output logs
+4. Stop the instance
+
+While the container instance is running (or after), you can either use the Azure Portal or the Azure CLI to check the benchmark results with:
+
+```bash
+az container logs -g $resourceGroupName -n cosmosdbsdkperf
+```
+
+Additionally you can check the logs in the Azure Portal by navigating to the container instance and clicking on the **Logs** tab.
+
+The logs will show the information, including the initial parameters:
+
+![Initial benchmark parameters](./arm2.png)
+
+And the results:
+
+![Benchmark results](./arm3.png)
+
+### Clean up
+
+If you want to remove the Benchmark instance, ypu can delete the container from the Azure Portal. You can also do so from the Azure CLI:
+
+```bash
+az container delete -g $resourceGroupName -n cosmosdbsdkperf
+```
+
+You can also delete the Benchmark instance from the Azure Portal by navigating to the container instance and clicking on the **Delete** button.
+
+**Remember to delete the Database and Container** that were created for the Benchmark in your CosmosDB account if you did not use the `CLEANUPFINISH` parameter as `true`.

Microsoft.Azure.Cosmos.Samples/Tools/Benchmark/ARMTemplate/benchmarkTemplate.json

@@ -0,0 +1,154 @@
+{
+ "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
+ "contentVersion": "1.0.0.0",
+ "parameters": {
+ "endpoint": {
+ "type": "string",
+ "metadata": {
+ "description": "Your account endpoint"
+ }
+ },
+ "key": {
+ "type": "securestring",
+ "metadata": {
+ "description": "Your account key"
+ }
+ },
+ "throughput": {
+ "type": "string",
+ "metadata": {
+ "description": "The throughput for the container."
+ }
+ },
+ "documents": {
+ "type": "string",
+ "metadata": {
+ "description": "The amount of items to insert."
+ }
+ },
+ "parallelism": {
+ "type": "string",
+ "metadata": {
+ "description": "The degree of parallelism."
+ },
+ "defaultValue": "-1"
+ },
+ "cleanUpOnFinish": {
+ "type": "string",
+ "metadata": {
+ "description": "Whether to delete the container after the benchmark is run."
+ },
+ "defaultValue": "false"
+ },
+ "containerGroupName": {
+ "type": "string",
+ "metadata": {
+ "description": "Name for the container group"
+ },
+ "defaultValue": "CosmosDBBenchmark"
+ },
+ "containerName": {
+ "type": "string",
+ "metadata": {
+ "description": "Name for the container"
+ },
+ "defaultValue": "cosmosdbsdkperf"
+ },
+ "volumeName": {
+ "type": "string",
+ "metadata": {
+ "description": "Name for the gitRepo volume"
+ },
+ "defaultValue": "cosmosdbsdk"
+ },
+ "port": {
+ "type": "string",
+ "metadata": {
+ "description": "Port to open on the container and the public IP address."
+ },
+ "defaultValue": "80"
+ },
+ "cpuCores": {
+ "type": "string",
+ "metadata": {
+ "description": "The number of CPU cores to allocate to the container."
+ },
+ "defaultValue": "4"
+ },
+ "memoryInGb": {
+ "type": "string",
+ "metadata": {
+ "description": "The amount of memory to allocate to the container in gigabytes."
+ },
+ "defaultValue": "8"
+ },
+ "location": {
+ "type": "string",
+ "defaultValue": "[resourceGroup().location]",
+ "metadata": {
+ "description": "Location for all resources."
+ }
+ }
+ },
+ "variables": {},
+ "resources": [
+ {
+ "name": "[parameters('containerGroupName')]",
+ "type": "Microsoft.ContainerInstance/containerGroups",
+ "apiVersion": "2020-11-01",
+ "location": "[parameters('location')]",
+ "properties": {
+ "containers": [
+ {
+ "name": "[parameters('containerName')]",
+ "properties": {
+ "image": "mcr.microsoft.com/dotnet/sdk:6.0",
+ "ports": [
+ {
+ "port": "[parameters('port')]"
+ }
+ ],
+ "resources": {
+ "requests": {
+ "cpu": "[parameters('cpuCores')]",
+ "memoryInGB": "[parameters('memoryInGb')]"
+ }
+ },
+ "volumeMounts": [
+ {
+ "name": "[parameters('volumeName')]",
+ "mountPath": "/mnt/gitrepos/",
+ "readOnly": false
+ }
+ ],
+ "command": [
+ "/bin/bash",
+ "-c",
+ "[concat('cd /mnt/gitrepos/azure-cosmos-dotnet-v3/Microsoft.Azure.Cosmos.Samples/Tools/Benchmark/ARMTemplate;export ENDPOINT=\"', parameters('endpoint'),'\";export KEY=\"', parameters('key'),'\";export THROUGHPUT=\"', parameters('throughput'),'\";export DOCUMENTS=\"', parameters('documents'),'\";export PARALLELISM=\"', parameters('parallelism'),'\";export CLEANUPFINISH=\"', parameters('cleanUpOnFinish'),'\";chmod +x run.sh;./run.sh')]"
+ ]
+ }
+ }
+ ],
+ "osType": "Linux",
+ "restartPolicy": "Never",
+ "ipAddress": {
+ "type": "Public",
+ "ports": [
+ {
+ "protocol": "TCP",
+ "port": "[parameters('port')]"
+ }
+ ]
+ },
+ "volumes": [
+ {
+ "name": "[parameters('volumeName')]",
+ "gitRepo": {
+ "repository": "https://github.com/Azure/azure-cosmos-dotnet-v3"
+ }
+ }
+ ]
+ }
+ }
+ ]
+}

Microsoft.Azure.Cosmos.Samples/Tools/Benchmark/ARMTemplate/parameters.json

@@ -0,0 +1,39 @@
+{
+ "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
+ "contentVersion": "1.0.0.0",
+ "parameters": {
+ "endpoint": {
+ "value": "<your-endpoint-here>"
+ },
+ "key": {
+ "value": "<your-key-here>"
+ },
+ "throughput": {
+ "value": "<container-throughput>"
+ },
+ "documents": {
+ "value": "<number-of-documents>"
+ },
+ "parallelism": {
+ "value": "-1"
+ },
+ "cleanUpOnFinish": {
+ "value": "false"
+ },
+ "containergroupname": {
+ "value": "CosmosDBBenchmark"
+ },
+ "containername": {
+ "value": "cosmosdbsdkperf"
+ },
+ "volumename": {
+ "value": "cosmosdbsdk"
+ },
+ "cpuCores": {
+ "value": "4"
+ },
+ "memoryInGb": {
+ "value": "8"
+ }
+ }
+}

Microsoft.Azure.Cosmos.Samples/Tools/Benchmark/ARMTemplate/run.sh

@@ -0,0 +1,2 @@
+cd ..
+dotnet run -c Release -e ${ENDPOINT} -k ${KEY} -t ${THROUGHPUT} -n ${DOCUMENTS} --pl ${PARALLELISM} --CleanupOnFinish ${CLEANUPFINISH} -w InsertV2BenchmarkOperation