{"id":2266,"date":"2026-10-20T08:00:00","date_gmt":"2026-10-20T08:00:00","guid":{"rendered":"https:\/\/veritasai.ai\/?p=2266"},"modified":"2026-10-20T14:30:08","modified_gmt":"2026-10-20T14:30:08","slug":"model-drift","status":"publish","type":"post","link":"https:\/\/veritasai.ai\/nl\/model-drift\/","title":{"rendered":"Model drift: why your AI changes after launch and how to catch it"},"content":{"rendered":"
\n

Kort gezegd:<\/strong> Model drift is when an AI model slowly gives worse or different answers after launch, even though nobody changed its code. The world, the data and the model underneath keep moving. You catch it by running the same tests on a fixed schedule, comparing the results with a baseline and keeping the evidence.<\/p>\n

Your AI passed every check on launch day. Six months later, it may not. Model drift rarely makes noise. It shows up as a strange answer to a customer or a rise in support tickets. Or an auditor asks how you know the model still works. If you don’t test it regularly, you find out last.<\/p>\n

What is model drift?<\/h2>\n

Model drift is the gap that grows between how an AI model behaved when you tested it and how it behaves today. The code can be exactly the same. What changes is everything around it. The questions people ask shift. The data the model reads gets updated. The meaning of a good answer moves, for example when a policy or a price changes. And if you use a large language model from a vendor, the vendor can update or replace that model while your app keeps running. Each change is small. Together they add up, and nobody gets an alert. That is why drift needs ongoing monitoring. A test you ran once tells you how the model behaved on that day. It tells you nothing about this week.<\/p>\n

There are a few common forms. Data drift means the inputs change: new products, new slang, new types of customer. Concept drift means the right answer changes, while the model still gives the old one. Model updates mean the system underneath you is different, even if the name on the contract is the same.<\/p>\n

Why do AI models drift after launch?<\/h2>\n

AI models drift because they learn from a snapshot of the past, while they work in a present that keeps changing. A model trained or tuned on last year’s questions has never seen this year’s. For apps built on a vendor’s language model there is an extra reason: the model itself can change. Researchers at Stanford and UC Berkeley tested two versions of GPT-4, from March and June 2023, on the same questions. On one task, spotting prime numbers, accuracy fell from 84% to 51%. That is what their study on how ChatGPT’s behaviour changes over time<\/a> found. Same product name, very different results. Your own setup adds more moving parts. Someone edits a system prompt. A document in your knowledge base gets replaced. A new tool gets connected. Each one can change what the model says.<\/p>\n

We see the same pattern that software testing has always shown. Even well-designed AI systems drift and misbehave over time. The difference with normal software is that an AI model can change its behaviour without a new release.<\/p>\n

How do you detect model drift?<\/h2>\n

You detect model drift by asking the model the same set of questions on a fixed schedule and comparing the answers with a baseline. The baseline is the result from a moment when you knew the model behaved well. If the scores drop, or the answers change in ways you didn’t expect, something has drifted. This only works if the test set stays fixed and covers what matters to you. That means accuracy on your core questions, but also refusals, bias, privacy and safety. A test set that changes every run can’t show a trend. Good drift detection also keeps the exact responses, not just a score. When something goes wrong, you want to see what the model actually said, on which date, to which question.<\/p>\n

A practical setup has four parts:<\/p>\n