Your AI Product Needs a Model Exit Plan
Design AI products so model changes, fallbacks, routing, pricing, and provider outages do not break the workflow.
The question most teams ask is, “Which model should we use?”
The better question is, “What happens when that answer changes?”
Models improve, route differently, get restricted, change price, change latency, or become unavailable in a workflow that used to work yesterday. That does not make AI products impossible. It means the model cannot be treated as invisible plumbing.
Production AI systems need a model exit plan.
A model name is not a contract
In normal software, a dependency has an API. You pin a version, test an upgrade, and decide when to move.
Frontier models are softer than that. The name may stay the same while behavior changes underneath. A provider can update weights, add safety routing, shift rate limits, change pricing, restrict access, or route a request to another model.
If your product only knows “call this model,” you do not really own the workflow.
You own the workflow when you know what quality, latency, cost, format, and policy behavior the task requires.
Design around capabilities
Do not scatter raw model calls across the app.
Create task-level capabilities:
extract_invoice_fieldsdraft_customer_replyclassify_support_ticketsummarize_brokerage_statementpropose_code_change
The product calls the capability. The capability chooses the current model, prompt, schema, tools, and fallback.
That seam is boring. It is also what lets you change providers later without rewriting the product.
Write the contract before the fallback
Before you design fallback logic, write the workflow contract.
What is the model allowed to do? What output shape is required? What makes the answer invalid? What is the latency budget? What data can leave the system? What happens if confidence is low?
If the task is low risk, maybe a cheaper model is fine. If the task touches money, compliance, health, or identity, maybe fallback should be human review instead of another model.
Not every fallback should be automatic.
Evals are portability
You cannot switch models safely if you do not know what good looks like.
That is what evals are for.
A useful eval is not a leaderboard. It is a representative set of product cases: messy documents, weird customer language, edge-case records, expected output, failure rules, and acceptance thresholds.
When a model changes, run the workflow again. If the results still pass, you can move. If not, you know where the product is exposed.
Evals are how you turn model choice from opinion into engineering.
Keep the user experience honest
Model failure should not always look like product failure.
Sometimes the right response is, “We need review before completing this.” Sometimes it is, “This document could not be processed with enough confidence.” Sometimes it is, “This step is queued because the preferred capability is unavailable.”
That message is part of the product.
Users can tolerate limits when the system explains them clearly. They lose trust when a product pretends a risky answer is complete.
How this connects to loops
This is the infrastructure cousin of loop engineering.
Loop engineering asks how an AI workflow pursues an outcome, verifies progress, and knows when to stop. A model exit plan asks what happens when one capability inside that loop changes.
The principle is the same: own the system around the model.
Model choice matters. But product resilience matters more.