Guides · Reliability

The model deprecation playbook: survive your provider's next retirement

Every model your product depends on has a retirement date; most providers will tell you, some will tell you late, and a few changes will never be announced at all. The teams that get hurt are not the ones without a migration plan — they are the ones who learn about the deadline from their error logs. This playbook is how to hear it from a calendar instead.

By Ingenious AI · Published 1 November 2026

The three ways a model changes under you

Announced deprecation. The provider publishes a shutdown date for a model version, typically with months of runway — if you see the notice. Notices live in changelogs, developer forums, dashboard banners and emails to the account owner (who left the company in March). The failure mode is organisational, not technical.
Alias drift. You call a floating name — "latest", an unpinned family alias — and the provider points it at new weights. Nothing "breaks"; outputs get subtly different: tone shifts, JSON gets sloppier, a refusal boundary moves. Your dashboards stay green while output quality walks away.
Behaviour change on a pinned version. Safety tuning, system-prompt changes and infrastructure updates can alter a versioned model's behaviour without a version bump. Rare, real, and the reason "we pinned it" is the start of the answer, not the end.

Detection: know before your users do

Watch the sources. Subscribe to each provider's changelog, deprecation page and status feed — and keep watching their terms and policy pages, where retirement language often lands first. Assign a named owner per provider; "the team" owns nothing.
Keep a model inventory. Every model identifier in your codebase, config and vendor consoles, with the feature it powers and the date you last verified it. You cannot migrate what you cannot list — and shadow usage (the prototype that shipped) is where inventories earn their keep.
Run golden-set evals continuously. A fixed set of representative prompts with scored expected behaviour, run against production models on a schedule. Drift shows up as a score change days before it shows up as a complaint. Keep the set small, representative and ruthless about covering your money features: extraction accuracy, format compliance, refusal behaviour, tool-call correctness.
Log model identifiers with every call. When quality dips, the first question is "what changed?" — request logs that carry the resolved model version let you answer in minutes (and belong in your audit trail for agent actions).

ModelMinder exists because we kept re-learning the first two bullets by hand: it watches the models your product depends on and alerts you before a deprecation or behaviour change breaks a release. The watch is automatable; the judgement calls below are not.

The migration drill

Pin deliberately. Production calls use versioned model IDs, never floating aliases. Aliases are for experiments. Record the pin and its review date in the inventory.
Abstract the provider boundary. One internal interface for "the model call," with provider specifics behind it. A migration should be a config change plus an eval run, not a refactor.
Gate on evals, not vibes. The candidate model runs the golden set plus a shadow slice of real traffic; it ships when scores hold, with special attention to the tails — the longest outputs, the weirdest inputs, the refusal edges.
Stage the cutover. Percentage rollout with the old model one flag-flip away. Deprecation deadlines argue for starting early enough that staging is still possible; the last week of a shutdown window is when everyone else's migration traffic hits the same new model.
Keep a fallback warm. For your most critical feature, a second provider (or a second model family) already integrated and eval-passed. Cold fallbacks fail in exciting ways at 2 a.m.; warm ones are a routing change.
Write the post-mortem before the incident. One page: who owns model changes, where the inventory lives, how the eval gate runs, what the rollback is. Print it. The 2 a.m. version of you will be grateful.

The calendar view

Put every known retirement date on one calendar, reviewed monthly alongside vendor-terms diffs and eval scores. Three dates matter per model: the announcement, your migration deadline (weeks before theirs), and the shutdown. Teams that run this rhythm stop experiencing deprecations as emergencies; a retirement becomes what it should have been all along — a scheduled maintenance task with excellent notice. The model market will keep moving at its current pace, which is to say: faster than your release cycle, forever. The playbook is not about predicting the moves. It is about never being surprised by one.