Getting Started with Microsoft.Extensions.AI: One Interface for Every LLM in .NET

Here's a problem every team building AI features hits within six months: you wrote your code against one provider's SDK, and now you want to switch. Maybe a better model shipped, maybe pricing changed, maybe compliance wants Azure-hosted only, maybe you want free local models in dev instead of burning API credits. Suddenly you're grepping for every SDK call in the codebase.

.NET has a first-party answer: Microsoft.Extensions.AI — a set of abstractions (IChatClient, IEmbeddingGenerator) that do for LLMs what ILogger did for logging. Code against the interface; plug in OpenAI, Azure OpenAI, Ollama, or anything else behind it.

The core idea: IChatClient

dotnet add package Microsoft.Extensions.AI
dotnet add package OpenAI          # one possible provider
dotnet add package Microsoft.Extensions.AI.OpenAI  # its adapter

using Microsoft.Extensions.AI;
using OpenAI;

IChatClient client = new OpenAIClient(apiKey)
    .GetChatClient("gpt-4o-mini")
    .AsIChatClient();

ChatResponse response = await client.GetResponseAsync(
[
    new ChatMessage(ChatRole.System, "You are a concise assistant."),
    new ChatMessage(ChatRole.User, "Explain dependency injection in one paragraph."),
]);

Console.WriteLine(response.Text);

Nothing in your application code below this point mentions OpenAI. Which means this also works — same interface, zero application changes — for a local model via Ollama:

IChatClient client = new OllamaApiClient(
    new Uri("http://localhost:11434"), "llama3.1");

Or Azure OpenAI, or an in-memory fake in your unit tests. This is exactly the seam I've told every team to hand-roll around vendor SDKs — except now it's standard, which means libraries can share it too (Semantic Kernel accepts IChatClient, vector stores accept IEmbeddingGenerator, and so on).

Streaming is part of the contract

await foreach (ChatResponseUpdate update in
    client.GetStreamingResponseAsync(messages, cancellationToken: ct))
{
    Console.Write(update.Text);
}

IAsyncEnumerable all the way down — it composes naturally with SSE or SignalR for the front end, and every provider adapter supports it.

The killer feature: middleware pipelines

Because everything is one interface, cross-cutting concerns become decorators — the same pattern you know from ASP.NET Core middleware. The library ships the important ones:

builder.Services.AddChatClient(services =>
        new OpenAIClient(config["OpenAI:ApiKey"])
            .GetChatClient("gpt-4o-mini")
            .AsIChatClient())
    .UseDistributedCache()   // identical prompts → cached responses
    .UseFunctionInvocation() // automatic tool calling (more below)
    .UseOpenTelemetry();     // traces + token usage in your observability stack

Think about what that caching line alone does for a feature like "summarize this product description": repeat requests cost you nothing and return instantly. And OpenTelemetry integration means token usage and latency show up in the same dashboards as the rest of your system — no bespoke logging layer.

You can write your own middleware, too — rate limiting per tenant, PII scrubbing, prompt logging — as a DelegatingChatClient in the pipeline.

Tool calling without the ceremony

The raw function-calling flow (model returns "call this function", you invoke it, send back the result, repeat) is boilerplate-heavy. UseFunctionInvocation runs that loop for you:

[Description("Gets the current stock level for a product SKU.")]
static int GetStockLevel(string sku) => InventoryDb.Lookup(sku);

var options = new ChatOptions
{
    Tools = [AIFunctionFactory.Create(GetStockLevel)]
};

ChatResponse response = await client.GetResponseAsync(
    "How many units of SKU-12345 do we have?", options);
// The middleware called GetStockLevel and fed the result back automatically.

AIFunctionFactory.Create reads your method signature and [Description] attributes to build the JSON schema the model sees. Strongly-typed C# in, working tool call out. (For the underlying mechanics and the security caveats you should know before exposing tools, see the function-calling deep dive linked below.)

Embeddings, same story

The second abstraction covers vector embeddings for semantic search and RAG:

IEmbeddingGenerator<string, Embedding<float>> generator =
    new OpenAIClient(apiKey)
        .GetEmbeddingClient("text-embedding-3-small")
        .AsIEmbeddingGenerator();

var embeddings = await generator.GenerateAsync(
    ["How do I reset my password?", "Billing questions"]);

Swap in a local embedding model for dev, a hosted one for prod — consuming code unchanged.

When to use it (and when not to)

Use Microsoft.Extensions.AI when:

• You're building application features on chat/embedding models — this should be your default seam in any new .NET AI code.
• You want dev/test against local or fake models and prod against hosted ones.
• You want caching/telemetry/tool-calling without hand-rolling infrastructure.

Go provider-direct when:

• You need a provider-exclusive capability the abstraction doesn't surface yet (bleeding-edge API features tend to arrive in native SDKs first). Even then: isolate that call, keep the rest on IChatClient.

Semantic Kernel? It sits above this layer — orchestration, agents, plugins, planning. If you need workflows and multi-step agent logic, use SK (it builds on these same abstractions). If you need "call a model, maybe with tools," Microsoft.Extensions.AI alone is lighter and plenty.

My recommendation

Treat IChatClient the way you treat ILogger: the obvious, boring, correct default. The AI provider landscape reshuffles every quarter; the teams that coded against an abstraction ride the reshuffles with a config change, and the teams that didn't get a refactoring sprint. Be the first team.

Where to go next

• How to Call the OpenAI API from C# — the raw fundamentals underneath the abstraction.
• Running Local LLMs with Ollama and C# — the local-model setup that pairs perfectly with IChatClient.
• LLM Function Calling in C# — what UseFunctionInvocation is doing for you, and how to do it safely.