Stop Parsing LLM Prose: Design AI Features as Typed Decisions
The most common AI feature we are asked to fix looks like this: one big prompt, a long text answer, and a parser that tries to pull a decision out of the prose. It works in the demo. In production the model adds a polite preface, changes a label's casing, or explains instead of answering, and the parser quietly picks the wrong branch.
We design AI features the other way around. Code owns the workflow. The model supplies small, typed judgments at the points where ordinary code needs semantic understanding: which handler, which candidate, how much, does this claim hold. This article is the pattern set we use, with notes on when each one fits.
Start from the behavior, work back to the judgments
Ask what the application will show, select, change, or hand off. Then list the decisions it needs along the way. A support inbox does not need "an AI that handles tickets". It needs to know the intent, whether the customer is angry, whether an order number is present and which one it is, and whether the drafted answer is supported by the policy.
Each of those is a judgment with a defined answer type: one of a set, a yes or no probability, or a position on a scale. Everything else, including lookups, calculations, and the actual refund, stays in code. Known rules belong in code too. If refunds over a threshold always need approval, that is an if statement, not a sentence in a prompt.
Route, and fill the known arguments
Routing is the simplest typed decision: pick one handler from a defined list, and include a "none of these" option so the model is never forced into a wrong branch. Where a branch needs parameters, ask for them up front as typed fields, so code receives a handler name and arguments rather than a paragraph.
Independent questions about the same input can run together in parallel. Ask the intent and the urgency and the language at once, and let code consume only the answers it needs. Ask a second round only when an earlier answer decides what evidence to fetch next.
Select instead of generate
Generation is where hallucinations live. When the answer already exists in the source, do not ask the model to write it. Find the candidates in code (every date, every amount, every order number pattern in the email), then ask the model to choose which candidate is the intended one, and copy that value exactly.
This removes a whole class of errors. The model cannot invent an invoice total that is not in the document, because it is only allowed to point at one. Check candidate coverage, though: the model cannot pick a value your extraction step missed. Include a no-match option and log how often it fires.
If you have a workflow today that runs on prompt-and-parse and breaks in ways nobody can reproduce, this redesign is usually the first fix we make in an automation build.
Want this automated for you?
One workflow in your real tools in about two weeks, with a human checkpoint.
Score once, decide in code
For ranking and triage, ask for scores on described dimensions (relevance, urgency, fit) with concrete levels that stand on their own. Store the raw scores. Then let code apply weights, thresholds, and filters. Changing a weight or a cutoff should not require running inference again.
Keep policy explicit. Weighted averages suit preferences that can trade off against each other. An "any serious violation blocks this" rule is not a weighted average. It needs its own separate yes or no check, so a high score elsewhere can never hide it.
Verify, and escalate the uncertain cases
For anything customer-facing, add a verification judgment after the main step: is this field supported by the source span, does this answer follow from the cited policy. Cases that fail, or pass with low confidence, go to a person or a heavier reasoning model. Everything else proceeds.
This is how we keep a human checkpoint on anything that emails a customer or moves money without making the human read every case. They read the cases the system is unsure about, which is the work worth a human's time.
Confidence thresholds, set on your data
Probabilities are useful only if you set thresholds against real outcomes. Take a few hundred representative cases, run them, and pick the cutoff where the cost of a wrong automatic action matches the cost of a human review. A refund decision and a tag suggestion should not share a threshold.
Read confidence carefully. A yes or no probability near 0.5 means the model sees both as similarly likely, not "medium". Low confidence on a harmless preference can be fine. And typed output guarantees the interface, not the truth. Test the end-to-end behavior, and when something fails, separate missing evidence, model error, code error, and service failure before you change anything.
Tooling choices
You can build this pattern with structured outputs or function calling from any major model provider, with a schema validator on your side. There are also models built specifically for typed judgments; TypeSafe is one option we have looked at. The tool matters less than the discipline: small questions, defined answers, rules in code, thresholds on data.
When not to bother: if the feature really is open-ended writing, such as a first draft of a blog post, generation is the job. Typed decisions are for the places where your code needs to branch.
Key takeaways
- Code owns the workflow. The model answers small questions with defined answer types.
- Keep known rules, calculations, and lookups in code, never in the prompt.
- Select from candidates found in code instead of generating values, and include a no-match option.
- Store raw scores and apply weights and thresholds in code, so policy changes do not need re-inference.
- Verify customer-facing outputs and escalate low-confidence cases to a human.
Frequently asked questions
It fixes the format, not the design. A single giant JSON answer still mixes many judgments together. Splitting it into narrow questions, each with defined options, is what makes the behavior testable.
Run a few hundred real cases with known outcomes and pick the cutoff where a wrong automatic action costs about the same as sending the case to a human. Set separate thresholds for decisions with different consequences.
Sometimes there are more calls, but each is small and many run in parallel. Measure actual cost per task and latency on your workload rather than assuming either way.
When the output really is free-form writing for a human to read, or when the decision is simple enough for ordinary code. Not every feature needs a model.
The 15-Day AI MVP Playbook
Day-by-day plan, a starter CLAUDE.md, the Lovable-to-production checklist, eval and cost worksheets, and 10 prompts. Pure value, no pitch.
Book 20 minutes, or send a brief
A former CTO replies. You leave with a written plan and a fixed estimate.
Prefer LinkedIn? Connect with Harish.