What You Built in This Module

“The pipeline passed” and “the model improved” are different claims, and a green build is the first one masquerading as the second. The whole point of MLOps is a system that refuses to ship a passing pipeline until it has proven the model is actually better — because for an ML system the data can drift, a feature can leak, and a retrained model can be worse, and none of those is a code bug a unit test would catch. What you can now change in production tests the data, gates on the right metric, versions every model so it is reversible, and exposes a new version to a sliver of traffic before all of it.

What you completed

A green pipeline came first; each lesson after it added a check for something a passing build cannot tell you, ending on the loop that can degrade itself while every metric looks fine:

• Passing on a fresh runner is what separates CI from “works on my machine.” A pipeline is a compiled graph of isolated jobs, not a script, and the value is that it runs on a clean machine that carries none of your local state. Ordering stages cheapest-failure-first is what makes that feedback fast instead of a twenty-minute wait for a typo to surface.
• A code-only test suite passes while bad data quietly degrades the model. The data is a second input, and testing schema, types, ranges, and nullability at the boundary is the only place CI can catch the leakage that makes a model look brilliant in validation and fail in production. No assertion about the code would ever see it.
• A green build is not a good model. Gating promotion on a metric that survives class imbalance — not accuracy, which a majority-class guess can ace on a loan book — is what stops a worse model from shipping because the build went green. Versioning each model as artifact plus the data snapshot and seed that produced it is what makes it auditable and reversible rather than merely deployed.
• Automatic retraining is how a model keeps up with drift and how it poisons itself. Train on labels its own predictions influenced, or on data a feedback loop has skewed, and every cycle degrades the model while every metric you log looks fine. The guards are what let the loop chase drift without slowly turning the system against itself.
• A model that passed every gate can still be wrong on live traffic. Shadowing a new version on the real stream with zero blast radius, canarying it to a bounded slice, then going full is what catches that on a sliver instead of all of it. Standardizing the path is what keeps a deploy that once took months a three-day deploy — the discipline, not the heroics.

Check your understanding

Work these without scrolling back up. Each one is a new system, not the loan-scoring pipeline the lessons used — the point is to apply the judgment, not recall the case. Try to answer before opening the hint.

1. A spam-classification pipeline runs as a single shell script on each engineer’s laptop, and a teammate insists CI is the same thing run on a server. Explain what a real CI pipeline is that the script is not, and why ordering the stages cheapest-failure-first changes the feedback you get.

   Hint

   Passing on a fresh runner is what separates CI from "works on my machine." Think about a compiled graph of isolated jobs running on a clean machine that carries none of your local state, and why a typo should surface in seconds rather than after a twenty-minute build.

2. The spam pipeline has a thorough unit-test suite and it is all green, yet a bad upstream feed quietly shifted a feature’s encoding and the model has been degrading. Explain why no code test catches this, and what you would test at the boundary so CI sees it.

   Hint

   The data is a second input, and no assertion about the code would ever see a bad one. Think about testing schema, types, ranges, and nullability at the boundary as the only place CI can catch the leakage that makes a model look brilliant in validation and fail live.

3. A new spam model makes the build go green and a teammate wants to promote it on that basis, reporting 97% accuracy on a stream that is 3% spam. Explain why a green build is not a good model, the metric you would gate promotion on instead, and what you would version alongside the model so it is reversible.

   Hint

   A majority-class guess aces accuracy at this base rate. Think about gating on a metric that survives class imbalance, and versioning each model as the artifact plus the data snapshot and seed that produced it so it is auditable and reversible.

4. The spam classifier retrains automatically each week on traffic its own predictions shaped — messages it flagged get less engagement, which feeds back into the next training set. Every logged metric looks fine while the model slowly degrades. Explain the mechanism that poisons the loop and the kind of guard that lets retraining chase drift without it.

   Hint

   Automatic retraining is how a model keeps up with drift and how it poisons itself. Think about training on labels its own predictions influenced, why every metric you log still looks fine, and what guard breaks the loop's grip on its own training data.

5. A new spam model passes every offline gate, and the team wants to push it to all traffic at once. Describe the staged rollout that catches a model that is wrong only on live traffic before it hits everyone, and what standardizing that path buys an organization.

   Hint

   A model that passed every gate can still be wrong on live traffic. Think about shadowing on the real stream with zero blast radius, then canarying to a bounded slice, then going full, and why standardizing the path turns a months-long deploy into a three-day one.

What it adds up to

The pipeline passing is necessary; the model improving is the point. The system you can now change in production refuses to ship the first without proving the second — it tests the data, gates on the right metric, versions every model so it is reversible, and exposes a new version to a sliver of traffic before all of it. That is the difference between an ML system you are afraid to touch and one that improves itself safely on a schedule.

What you have built across the track

This is the last module, and the loop it closes is the loop the whole track has been building. You started with Python that was a script and made it maintainable software; turned messy data into a deterministic feature pipeline; learned when a model result is trustworthy and trained one that beats baseline; packaged it to run identically anywhere; served it behind an API; put it in a human’s hands; deployed it to a real URL; declared its infrastructure in code; made it observable; and now made it improve itself safely. The thread that ran through every module was the same one: a thing that works once is a demo, and a thing that works reliably, reproducibly, and safely for other people is the job. You have built the second one, end to end.

What remains is the capstone: the same arc, shipped solo, on a dataset and an API you have never used — no starter repo, no spec, no task list. The module projects proved you can build each piece beside a worked example; the capstone is the proof the whole thing transferred.