Part 1 of a series on building production AI on .NET — drawn from TextStack, a reader with seven shipping AI features.
You can build an AI feature in an afternoon. Wiring up an API call and a prompt is genuinely easy now. The hard part — the part that separates a demo from a product — is answering one deceptively simple question:
Is it any good? And did my last change make it better or worse?
For normal code, that question has a normal answer: a test suite. Add(2, 2) should return 4; if it doesn't, the build goes red. But an AI feature doesn't return 4. Ask it to explain a word and it returns a paragraph — a slightly different paragraph every single time, and "correct" is a whole range of good answers, not one. You cannot write Assert.Equal against prose. The thing software engineering relies on most — a fast, automatic signal that something broke — is gone.
Evals are how you get that signal back. This post is a plain-English introduction to what they are and how we actually run them in production. No hype, no notebooks — just the mental model and a real implementation.
