Releasing a new model to a small slice of live traffic, say 1 to 5%, watching errors, latency and business metrics, and widening the rollout in steps while they hold. If they don't, traffic goes back to the old model.
Card 1 of 228. Answer: Releasing a new model to a small slice of live traffic, say 1 to 5%, watching errors, latency and business metrics, and widening the rollout in steps while they hold. If they don't, traffic goes back to the old model.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
A canary deployment exposes a new version to a small share of real users first, so a bad release hurts few of them and is rolled back quickly. The name comes from the canaries miners carried to detect gas. The new model serves real predictions, unlike in a shadow deployment, so it can catch problems that only appear when the outputs are used.
Example: a rollout goes 1%, 10%, 50%, 100% over a few days, with automatic rollback if the error rate or 99th-percentile latency crosses a threshold. At 1% of 2 million daily requests the canary gets 20,000 requests a day: plenty to catch crashes, timeouts and broken features, but far too few to detect a small change in conversion rate.
That is the difference interviewers look for. A canary is a safety check that catches breakage early; an A/B test is an experiment, with randomized groups and a sample size planned to measure an effect. Related terms: blue-green deployment switches all traffic between two full environments at once, with instant rollback, and sticky routing keeps a user on one model so their experience doesn't flip between versions.
