Aspire Azure AI Inference integration (Preview)

Includes: Client integration only — Hosting integration not included

The Aspire Azure AI Inference integration provides a seamless way to deploy and manage machine learning models in the cloud. This integration allows you to leverage the power of Azure's AI services while maintaining the flexibility and ease of use of the Aspire.

Hosting integration

Although the Azure AI Inference library doesn't currently offer direct hosting integration, you can still integrate it into your AppHost project. Simply add a connection string to establish a reference to an existing Azure AI Foundry resource.

Connect to an existing Azure AI Foundry service

If you already have an Azure AI Foundry service, you can easily connect to it by adding a connection string to your AppHost. This approach uses a simple, string-based configuration. To establish the connection, use the AddConnectionString method:

var builder = DistributedApplication.CreateBuilder(args);

var aiFoundry = builder.AddConnectionString("ai-foundry");

builder.AddProject<Projects.ExampleProject>()
       .WithReference(aiFoundry);

// After adding all resources, run the app...

The connection string is configured in the AppHost's configuration, typically under User Secrets, under the ConnectionStrings section:

{
  "ConnectionStrings": {
    "ai-foundry": "Endpoint=https://{endpoint}/;DeploymentId={deploymentName}"
  }
}

For more information, see Add existing Azure resources with connection strings.

Client integration

To get started with the Aspire Azure AI Inference client integration, install the 📦 Aspire.Azure.AI.Inference NuGet package in the client-consuming project, that is, the project for the application that uses the Azure AI Inference client.

dotnet add package Aspire.Azure.AI.Inference

<PackageReference Include="Aspire.Azure.AI.Inference"
                  Version="*" />

For more information, see dotnet add package or Manage package dependencies in .NET applications.

Add an Azure AI Inference client

In the Program.cs file of your client-consuming project, use the AddAzureChatCompletionsClient method on any IHostApplicationBuilder to register an ChatCompletionsClient for dependency injection (DI).

builder.AddAzureChatCompletionsClient(connectionName: "ai-foundry");

After adding the ChatCompletionsClient, you can retrieve the client instance using dependency injection:

public class ExampleService(ChatCompletionsClient client)
{
    // Use client...
}

For more information, see:

• What is Azure AI model inference? for details on Azure AI model interfence.
• Dependency injection in .NET for details on dependency injection.
• The Azure AI Foundry SDK: C#.

Add Azure AI Inference client with registered IChatClient

If you're interested in using the IChatClient interface with the Azure AI Inference client, simply chain either of the following APIs to the AddAzureChatCompletionsClient method:

• AddChatClient: Registers a singleton IChatClient in the services.
• AddKeyedChatClient: Registers a keyed singleton IChatClient in the services.

For example, consider the following C# code that adds an IChatClient to the DI container:

builder.AddAzureChatCompletionsClient(connectionName: "ai-foundry")
       .AddChatClient("deploymentName");

Similarly, you can add a keyed IChatClient with the following C# code:

builder.AddAzureChatCompletionsClient(connectionName: "ai-foundry")
       .AddKeyedChatClient("serviceKey", "deploymentName");

After adding the IChatClient, you can retrieve the client instance using dependency injection:

public class ExampleService(IChatClient chatClient)
{
    public async Task<string> GetResponseAsync(string userMessage)
    {
        var response = await chatClient.CompleteAsync(userMessage);
        return response.Message.Text ?? string.Empty;
    }
}

For more information on the IChatClient and its corresponding library, see Artificial intelligence in .NET (Preview).

Add keyed Azure AI Inference clients

There might be situations where you want to register multiple ChatCompletionsClient instances with different connection names. To register keyed Azure AI Inference clients, call the AddKeyedAzureChatCompletionsClient method:

builder.AddKeyedAzureChatCompletionsClient(name: "chat");
builder.AddKeyedAzureChatCompletionsClient(name: "code");

Then you can retrieve the client instances using dependency injection. For example, to retrieve the clients from a service:

public class ExampleService(
    [KeyedService("chat")] ChatCompletionsClient chatClient,
    [KeyedService("code")] ChatCompletionsClient codeClient)
{
    // Use clients...
}

For more information, see Keyed services in .NET.

Configuration

The Aspire Azure AI Inference library provides multiple options to configure the Azure AI Foundry Service based on the requirements and conventions of your project.

Use a connection string

A connection can be constructed from the Keys, Deployment ID and Endpoint tab with the format:

Endpoint={endpoint};Key={key};DeploymentId={deploymentId}`

You can provide the name of the connection string when calling builder.AddAzureChatCompletionsClient():

builder.AddAzureChatCompletionsClient(
    connectionName: "connection-string-name");

The connection string is retrieved from the ConnectionStrings configuration section. Two connection formats are supported:

Azure AI Foundry endpoint

The recommended approach is to use an Endpoint, which works with the ChatCompletionsClientSettings.Credential property to establish a connection. If no credential is configured, DefaultAzureCredential is used.

{
  "ConnectionStrings": {
    "connection-string-name": "Endpoint=https://{endpoint}/;DeploymentId={deploymentName}"
  }
}

Connection string

Alternatively, a custom connection string can be used.

{
  "ConnectionStrings": {
    "connection-string-name": "Endpoint=https://{endpoint}/;Key={account_key};DeploymentId={deploymentName}"
  }
}

Use configuration providers

The Aspire Azure AI Inference library supports Microsoft.Extensions.Configuration. It loads the ChatCompletionsClientSettings and AzureAIInferenceClientOptions from configuration by using the Aspire:Azure:AI:Inference key. For example, consider an appsettings.json that configures some of the options:

{
  "Aspire": {
    "Azure": {
      "AI": {
        "Inference": {
          "DisableTracing": false,
          "EnableSensitiveTelemetryData": false,
          "ClientOptions": {
            "UserAgentApplicationId": "myapp"
          }
        }
      }
    }
  }
}

Use inline delegates

You can also pass the Action<ChatCompletionsClientSettings> configureSettings delegate to set up some or all the options inline, for example, to disable tracing from code:

builder.AddAzureChatCompletionsClient(
    connectionName: "connection-string-name",
    static settings => settings.DisableTracing = true);

Observability and telemetry

Aspire integrations automatically set up Logging, Tracing, and Metrics configurations, which are sometimes known as the pillars of observability. For more information about integration observability and telemetry, see Aspire integrations overview. Depending on the backing service, some integrations may only support some of these features. For example, some integrations support logging and tracing, but not metrics. Telemetry features can also be disabled using the techniques presented in the Configuration section.

Logging

The Aspire Azure AI Inference integration uses the following log categories:

• Azure.Core
• Azure.Identity

Tracing

The Aspire Azure AI Inference integration will emit the following tracing activities using OpenTelemetry:

• Experimental.Microsoft.Extensions.AI - Used by Microsoft.Extensions.AI to record AI operations

Configuring sensitive data in telemetry

By default, telemetry includes metadata such as token counts, but not raw inputs and outputs like message content. To include potentially sensitive information in telemetry, set the EnableSensitiveTelemetryData configuration option:

builder.AddAzureChatCompletionsClient(
    connectionName: "ai-foundry",
    configureSettings: settings =>
    {
        settings.EnableSensitiveTelemetryData = true;
    })
    .AddChatClient("deploymentName");

Or through configuration:

{
  "Aspire": {
    "Azure": {
      "AI": {
        "Inference": {
          "EnableSensitiveTelemetryData": true
        }
      }
    }
  }
}

Alternatively, you can enable sensitive data capture by setting the environment variable:

OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true

Using underlying library telemetry

If you need to access telemetry from the underlying Azure AI Inference library directly, you can manually add the appropriate activity sources and meters to your OpenTelemetry configuration:

builder.Services.AddOpenTelemetry()
    .WithTracing(tracing => tracing.AddSource("Azure.AI.Inference.*"))
    .WithMetrics(metrics => metrics.AddMeter("Azure.AI.Inference.*"));

However, you'll need to enable experimental telemetry support in the Azure AI Inference library by setting the AZURE_EXPERIMENTAL_ENABLE_ACTIVITY_SOURCE environment variable to "true" or calling AppContext.SetSwitch("Azure.Experimental.EnableActivitySource", true) during app startup.

See also