Understanding Embeddings and Vector Search in .NET (With Real Code)

Every interesting AI feature that isn't a chatbot — semantic search, "related articles", duplicate detection, document clustering, RAG — runs on the same primitive: embeddings. And unlike most of the AI stack, embeddings are something you can fully understand, not just call. This post builds that understanding with runnable C#.

What an embedding actually is

An embedding model maps text to a fixed-length array of floats — typically 384 to 3,072 of them:

text
"How do I reset my password?"  →  [0.021, -0.135, 0.884, ..., 0.007]

The magic is in how it maps: texts with similar meaning land at nearby points. Think of it as coordinates in a "meaning space" with a few thousand dimensions, where the model has arranged language so that direction and distance encode semantics. "Reset my password" sits near "can't log in", and both sit far from "shipping costs to Cape Town".

Once meaning is geometry, hard language problems become easy math: is this similar to that? is a distance calculation.

Getting embeddings in .NET

csharp
using Microsoft.Extensions.AI;
using OpenAI;

IEmbeddingGenerator<string, Embedding<float>> generator =
    new OpenAIClient(apiKey)
        .GetEmbeddingClient("text-embedding-3-small")
        .AsIEmbeddingGenerator();

GeneratedEmbeddings<Embedding<float>> result = await generator.GenerateAsync(
[
    "How do I reset my password?",
    "I can't log into my account",
    "What does shipping cost to Cape Town?",
]);

float[] first = result[0].Vector.ToArray(); // 1536 floats

(The IEmbeddingGenerator abstraction means a local model via Ollama plugs in identically — useful for dev and for data that can't leave your network.)

Measuring similarity: cosine

The standard similarity measure is cosine similarity — the cosine of the angle between two vectors: 1.0 means same direction (same meaning), 0 means unrelated.

csharp
public static float CosineSimilarity(ReadOnlySpan<float> a, ReadOnlySpan<float> b)
{
    float dot = 0, magA = 0, magB = 0;
    for (int i = 0; i < a.Length; i++)
    {
        dot += a[i] * b[i];
        magA += a[i] * a[i];
        magB += b[i] * b[i];
    }
    return dot / (MathF.Sqrt(magA) * MathF.Sqrt(magB));
}

Running it on our three sentences:

text
"reset password" vs "can't log in"      → ~0.72   (related!)
"reset password" vs "shipping to CPT"   → ~0.18   (unrelated)

No shared keywords between the first pair — the similarity is purely semantic. That's the entire trick that keyword search (LIKE queries, full-text search) can't do, and it's why embeddings power modern search experiences.

(Performance aside: System.Numerics.Tensors ships a SIMD-accelerated TensorPrimitives.CosineSimilarity that's dramatically faster than the scalar loop above — use it in real code, write the loop once for understanding.)

Semantic search in 20 lines

csharp
public record Document(string Id, string Text, float[] Vector);

public class SemanticIndex(IEmbeddingGenerator<string, Embedding<float>> generator)
{
    private readonly List<Document> _docs = [];

    public async Task AddAsync(string id, string text)
    {
        var embedding = (await generator.GenerateAsync([text]))[0];
        _docs.Add(new Document(id, text, embedding.Vector.ToArray()));
    }

    public async Task<List<(Document Doc, float Score)>> SearchAsync(
        string query, int top = 5)
    {
        var q = (await generator.GenerateAsync([query]))[0].Vector.ToArray();

        return _docs
            .Select(d => (d, Score: CosineSimilarity(q, d.Vector)))
            .OrderByDescending(x => x.Score)
            .Take(top)
            .ToList();
    }
}

Index your FAQ articles, search with the user's actual question, return the top hits. This exact class — linear scan and all — has shipped in internal tools I've built. Ten thousand 1,536-dimension vectors scan in a few milliseconds; you do not need infrastructure to start.

When you do need a vector database

The in-memory approach hits walls in three predictable places: the index doesn't survive restarts, it doesn't scale past one machine, and linear scan eventually gets slow (hundreds of thousands of vectors). That's when you reach for a vector store, which gives you persistence, filtered queries ("only documents this tenant can see" — you'll need this sooner than you think), and approximate-nearest-neighbor indexes (HNSW) that search millions of vectors in milliseconds by accepting a tiny recall trade-off.

Realistic .NET options, in the order I'd consider them:

  • pgvector — an extension for the PostgreSQL you probably already run; embeddings become a column type, similarity becomes SQL. Least new infrastructure, works with EF Core.
  • Azure AI Search — if you're an Azure shop and want hybrid (keyword + vector) search managed for you.
  • Qdrant / dedicated stores — when vector search is the product and you need scale and tuning.

The pipeline shape stays identical — embed, store, search — so starting in-memory and migrating later is a contained refactor, not a rewrite.

Practical gotchas from production

  • Never mix embedding models. Vectors from different models (or model versions) live in different spaces; comparing them yields garbage. Store the model name next to your vectors, and re-embed everything when you upgrade.
  • Similarity scores are relative, not absolute. Don't hardcode "0.8 = match" — calibrate thresholds against your own data, per use case.
  • Embed at the right granularity. Whole documents blur meaning ("this 40-page PDF is about... everything"); single sentences lose context. Paragraph-to-section chunks hit the sweet spot for search.
  • Cache embeddings. They're deterministic per model — embedding the same text twice is pure waste. Content-hash as the cache key works well.

The mental model to keep

Embeddings turn "does this mean the same thing?" into a number you can sort by — and .NET gives you everything needed to use that: an abstraction to generate them, LINQ to search them, and a clear upgrade path when scale demands it. Master this one primitive and half the AI feature requests on your backlog become straightforward engineering.

Where to go next