|
| 1 | +--- |
| 2 | +title: .NET Aspire Azure AI Inference integration (Preview) |
| 3 | +description: Learn how to use the .NET Aspire Azure AI Inference integration to deploy and manage machine learning models in the cloud. |
| 4 | +ms.date: 05/14/2025 |
| 5 | +titleSuffix: '' |
| 6 | +--- |
| 7 | + |
| 8 | +# .NET Aspire Azure AI Inference integration (Preview) |
| 9 | + |
| 10 | +[!INCLUDE [includes-client](../includes/includes-client.md)] |
| 11 | + |
| 12 | +The .NET Aspire Azure AI Inference integration provides a seamless way to deploy and manage machine learning models in the cloud. This integration allows you to leverage the power of Azure's AI services while maintaining the flexibility and ease of use of the .NET Aspire. |
| 13 | + |
| 14 | +## Hosting integration |
| 15 | + |
| 16 | +Although the Azure AI Inference library doesn't currently offer direct hosting integration, you can still integrate it into your app host project. Simply add a connection string to establish a reference to an existing Azure AI Foundry resource. |
| 17 | + |
| 18 | +### Connect to an existing Azure AI Foundry service |
| 19 | + |
| 20 | +If you already have an [Azure AI Foundry](https://ai.azure.com/) service, you can easily connect to it by adding a connection string to your app host. This approach uses a simple, string-based configuration. To establish the connection, use the <xref:Aspire.Hosting.ParameterResourceBuilderExtensions.AddConnectionString%2A> method: |
| 21 | + |
| 22 | +```csharp |
| 23 | +var builder = DistributedApplication.CreateBuilder(args); |
| 24 | + |
| 25 | +var aiFoundry = builder.AddConnectionString("ai-foundry"); |
| 26 | + |
| 27 | +builder.AddProject<Projects.ExampleProject>() |
| 28 | + .WithReference(aiFoundry); |
| 29 | + |
| 30 | +// After adding all resources, run the app... |
| 31 | +``` |
| 32 | + |
| 33 | +[!INCLUDE [connection-strings-alert](../includes/connection-strings-alert.md)] |
| 34 | + |
| 35 | +The connection string is configured in the app host's configuration, typically under User Secrets, under the `ConnectionStrings` section: |
| 36 | + |
| 37 | +```json |
| 38 | +{ |
| 39 | + "ConnectionStrings": { |
| 40 | + "ai-foundry": "Endpoint=https://{endpoint}/;DeploymentId={deploymentName}" |
| 41 | + } |
| 42 | +} |
| 43 | +``` |
| 44 | + |
| 45 | +For more information, see [Add existing Azure resources with connection strings](../azure/integrations-overview.md#add-existing-azure-resources-with-connection-strings). |
| 46 | + |
| 47 | +## Client integration |
| 48 | + |
| 49 | +To get started with the .NET Aspire Azure AI Inference client integration, install the [📦 Aspire.Azure.AI.Inference](https://www.nuget.org/packages/Aspire.Azure.AI.Inference) NuGet package in the client-consuming project, that is, the project for the application that uses the Azure AI Inference client. |
| 50 | + |
| 51 | +### [.NET CLI](#tab/dotnet-cli) |
| 52 | + |
| 53 | +```dotnetcli |
| 54 | +dotnet add package Aspire.Azure.AI.Inference |
| 55 | +``` |
| 56 | + |
| 57 | +### [PackageReference](#tab/package-reference) |
| 58 | + |
| 59 | +```xml |
| 60 | +<PackageReference Include="Aspire.Azure.AI.Inference" |
| 61 | + Version="*" /> |
| 62 | +``` |
| 63 | + |
| 64 | +--- |
| 65 | + |
| 66 | +For more information, see [dotnet add package](/dotnet/core/tools/dotnet-add-package) or [Manage package dependencies in .NET applications](/dotnet/core/tools/dependencies). |
| 67 | + |
| 68 | +### Add an Azure AI Inference client |
| 69 | + |
| 70 | +In the _Program.cs_ file of your client-consuming project, use the `AddChatCompletionsClient` method on any <xref:Microsoft.Extensions.Hosting.IHostApplicationBuilder> to register an <xref:Azure.AI.Inference.ChatCompletionsClient> for dependency injection (DI). |
| 71 | + |
| 72 | +```csharp |
| 73 | +builder.AddChatCompletionsClient(connectionName: "ai-foundry"); |
| 74 | +``` |
| 75 | + |
| 76 | +> [!TIP] |
| 77 | +> The `connectionName` parameter must match the name used when adding the Azure AI Inference resource in the app host project. For more information, see [Connect to an existing Azure AI Foundry service](#connect-to-an-existing-azure-ai-foundry-service). |
| 78 | +
|
| 79 | +After adding the `ChatCompletionsClient`, you can retrieve the client instance using dependency injection: |
| 80 | + |
| 81 | +```csharp |
| 82 | +public class ExampleService(ChatCompletionsClient client) |
| 83 | +{ |
| 84 | + // Use client... |
| 85 | +} |
| 86 | +``` |
| 87 | + |
| 88 | +For more information, see: |
| 89 | + |
| 90 | +- [What is Azure AI model inference?](/azure/ai-foundry/model-inference/overview) for details on Azure AI model interfence. |
| 91 | +- [Dependency injection in .NET](/dotnet/core/extensions/dependency-injection) for details on dependency injection. |
| 92 | +- [The Azure AI Foundry SDK: C#](/azure/ai-foundry/how-to/develop/sdk-overview?tabs=sync&pivots=programming-language-csharp). |
| 93 | + |
| 94 | +### Add keyed Azure AI Inference clients |
| 95 | + |
| 96 | +There might be situations where you want to register multiple `ChatCompletionsClient` instances with different connection names. To register keyed Azure AI Inference clients, call the `AddKeyedAzureChatCompletionsClient` method: |
| 97 | + |
| 98 | +```csharp |
| 99 | +builder.AddKeyedAzureChatCompletionsClient(name: "chat"); |
| 100 | +builder.AddKeyedAzureChatCompletionsClient(name: "code"); |
| 101 | +``` |
| 102 | + |
| 103 | +> [!IMPORTANT] |
| 104 | +> When using keyed services, ensure that your Azure AI Inference resource configures two named connections, one for `chat` and one for `code`. |
| 105 | +
|
| 106 | +Then you can retrieve the client instances using dependency injection. For example, to retrieve the clients from a service: |
| 107 | + |
| 108 | +```csharp |
| 109 | +public class ExampleService( |
| 110 | + [KeyedService("chat")] ChatCompletionsClient chatClient, |
| 111 | + [KeyedService("code")] ChatCompletionsClient codeClient) |
| 112 | +{ |
| 113 | + // Use clients... |
| 114 | +} |
| 115 | +``` |
| 116 | + |
| 117 | +For more information, see [Keyed services in .NET](/dotnet/core/extensions/dependency-injection#keyed-services). |
| 118 | + |
| 119 | +### Configuration |
| 120 | + |
| 121 | +The .NET Aspire Azure AI Inference library provides multiple options to configure the Azure AI Foundry Service based on the requirements and conventions of your project. |
| 122 | + |
| 123 | +> [!NOTE] |
| 124 | +> Either an `Endpoint` and `DeploymentId`, or a `ConnectionString` is required to be supplied. |
| 125 | +
|
| 126 | +#### Use a connection string |
| 127 | + |
| 128 | +A connection can be constructed from the `Keys`, `Deployment ID` and `Endpoint` tab with the format: |
| 129 | + |
| 130 | +```Plaintext |
| 131 | +Endpoint={endpoint};Key={key};DeploymentId={deploymentId}` |
| 132 | +``` |
| 133 | + |
| 134 | +You can provide the name of the connection string when calling `builder.AddChatCompletionsClient()`: |
| 135 | + |
| 136 | +```csharp |
| 137 | +builder.AddChatCompletionsClient( |
| 138 | + connectionName: "connection-string-name"); |
| 139 | +``` |
| 140 | + |
| 141 | +The connection string is retrieved from the `ConnectionStrings` configuration section. Two connection formats are supported: |
| 142 | + |
| 143 | +##### Azure AI Foundry endpoint |
| 144 | + |
| 145 | +The recommended approach is to use an `Endpoint`, which works with the `ChatCompletionsClientSettings.Credential` property to establish a connection. If no credential is configured, <xref:Azure.Identity.DefaultAzureCredential> is used. |
| 146 | + |
| 147 | +```json |
| 148 | +{ |
| 149 | + "ConnectionStrings": { |
| 150 | + "connection-string-name": "Endpoint=https://{endpoint}/;DeploymentId={deploymentName}" |
| 151 | + } |
| 152 | +} |
| 153 | +``` |
| 154 | + |
| 155 | +##### Connection string |
| 156 | + |
| 157 | +Alternatively, a custom connection string can be used. |
| 158 | + |
| 159 | +```json |
| 160 | +{ |
| 161 | + "ConnectionStrings": { |
| 162 | + "connection-string-name": "Endpoint=https://{endpoint}/;Key={account_key};DeploymentId={deploymentName}" |
| 163 | + } |
| 164 | +} |
| 165 | +``` |
| 166 | + |
| 167 | +#### Use configuration providers |
| 168 | + |
| 169 | +The .NET Aspire Azure AI Inference library supports <xref:Microsoft.Extensions.Configuration>. It loads the `ChatCompletionsClientSettings` and `AzureAIInferenceClientOptions` from configuration by using the `Aspire:Azure:AI:Inference` key. For example, consider an _appsettings.json_ that configures some of the options: |
| 170 | + |
| 171 | +```json |
| 172 | +{ |
| 173 | + "Aspire": { |
| 174 | + "Azure": { |
| 175 | + "AI": { |
| 176 | + "Inference": { |
| 177 | + "DisableTracing": false, |
| 178 | + "ClientOptions": { |
| 179 | + "UserAgentApplicationId": "myapp" |
| 180 | + } |
| 181 | + } |
| 182 | + } |
| 183 | + } |
| 184 | + } |
| 185 | +} |
| 186 | +``` |
| 187 | + |
| 188 | +#### Use inline delegates |
| 189 | + |
| 190 | +You can also pass the `Action<ChatCompletionsClientSettings> configureSettings` delegate to set up some or all the options inline, for example, to disable tracing from code: |
| 191 | + |
| 192 | +```csharp |
| 193 | +builder.AddChatCompletionsClient( |
| 194 | + connectionName: "connection-string-name", |
| 195 | + static settings => settings.DisableTracing = true); |
| 196 | +``` |
| 197 | + |
| 198 | +[!INCLUDE [integration-observability-and-telemetry](../includes/integration-observability-and-telemetry.md)] |
| 199 | + |
| 200 | +### Logging |
| 201 | + |
| 202 | +The .NET Aspire Azure AI Inference integration uses the following log categories: |
| 203 | + |
| 204 | +- `Azure.Core` |
| 205 | +- `Azure.Identity` |
| 206 | + |
| 207 | +### Tracing |
| 208 | + |
| 209 | +The .NET Aspire Azure AI Inference integration emits tracing activities using OpenTelemetry for operations performed with the `OpenAIClient`. |
| 210 | + |
| 211 | +> [!IMPORTANT] |
| 212 | +> Azure AI Inference telemetry support is experimental, and the shape of traces may change in the future without notice. It can be enabled by invoking: |
| 213 | +> |
| 214 | +> ```csharp |
| 215 | +> AppContext.SetSwitch("Azure.Experimental.EnableActivitySource", true); |
| 216 | +> ``` |
| 217 | +> |
| 218 | +> Alternatively, you can set the `AZURE_EXPERIMENTAL_ENABLE_ACTIVITY_SOURCE` environment variable to `"true"`. |
| 219 | +
|
| 220 | +## See also |
0 commit comments