What You Built in This Module

An endpoint that answers one request and one that survives traffic look identical on the demo and behave nothing alike under load — the difference is concurrency, the input you did not validate, and the tail latency the median hides. Each of those is a failure the happy path sails past until production arrives. What you can run now is a service other systems can call without you watching it: it loads once, refuses input it should not trust, and fails in a bounded way instead of hanging.

What you completed

A working endpoint came first; each lesson after it introduced something real traffic does that a single request never reveals:

• The vocabulary trips people because it is taught before it is used. Request versus response, route versus handler, status code versus body — naming these is one thing, watching a 422 land in the status channel while the body carries the reason is another. Standing up the route and reading the response is what makes the rest of the module legible.
• A blocking call inside an async handler freezes everyone, not only the slow request. Loading the model in the request path, or running synchronous work on the event loop, is the bug that passes every single-request test and collapses the moment two requests overlap. Moving the load to startup and knowing which handler runs in a threadpool is what keeps one slow caller from taking down the rest.
• Validation at the door is also how a leak becomes impossible to send. An undeclared or wrong-typed field flows past a trusting handler and surfaces as a misleading 500 deep in the model, where the cause is hardest to find. The pydantic schema turns that into a clear 422 at the boundary — and the same schema is what makes a leaky feature structurally unsendable, the train/serve contract enforced at the edge.
• A batch path is only faster if the answers come back attached to the right inputs. Scoring many records in one call saves per-record overhead, but a misaligned batch returns confident predictions for the wrong rows with no error at all. Keeping the alignment exact and measuring what the batch path actually costs is what stops “faster” from meaning “faster and wrong.”
• The median lies about the tail, and the tail is what pages you. A service that looks fast at p50 can be timing out at p99 under the same load, and a model that dies should fail fast rather than hang every request waiting on it. Reading p50 against p99 and finding where one box stops being enough is the difference between a service that degrades gracefully and one that falls over silently.

Check your understanding

Work these without scrolling back up. Each one is a new service, not the loan-scoring endpoint the lessons used — the point is to apply the judgment, not recall the case. Try to answer before opening the hint.

1. You stand up a support-ticket triage endpoint and a teammate is confused that a request which “failed” still returned a body with text in it. Explain the separate roles of the status code and the response body, and what a 422 in the status channel tells a caller that the body does not.

   Hint

   The vocabulary trips people because the status and the body answer different questions. Think about one carrying the machine-readable outcome and the other carrying the human-readable reason, and what reading both together gives you.

2. The triage service loads its model from disk inside the async predict handler. It passes every single-request test, then falls over the moment two tickets arrive at once. Explain why a blocking call on the event loop freezes every caller and not only the slow one, and the two changes that fix it.

   Hint

   One synchronous call on the event loop blocks the whole loop. Think about moving the load to startup so it happens once, and knowing which kind of handler runs in a threadpool so synchronous work does not stall everyone.

3. The triage endpoint accepts a raw JSON body and one day receives a ticket with priority as a string and an undeclared extra field. The model raises a confusing 500 deep in its scoring code. Describe where you would catch this, what status the caller should see instead, and how the same boundary makes a leaky feature impossible to send.

   Hint

   An untyped body flows past a trusting handler and surfaces as a misleading error where the cause is hardest to find. Think about a schema at the door turning that into a clear rejection, and how declaring the allowed fields enforces the train/serve contract at the edge.

4. To cut overhead the triage service adds an endpoint that scores 500 tickets in one call. It returns fast, but a downstream system reports predictions attached to the wrong tickets with no error logged. Explain the failure a batch path introduces and what you would verify and measure before calling it faster.

   Hint

   A batch is only faster if the answers come back attached to the right inputs. Think about what keeps the ordering exact from input to output, and why "faster" is meaningless until you have measured what the batch path actually costs.

5. The triage service looks fast in the dashboard at its median response time, yet a downstream team reports requests timing out under the same load. Explain why the median hides this, which percentile you would read instead, and how a model that dies should fail so it does not take the whole service with it.

   Hint

   The median lies about the tail, and the tail is what pages you. Think about reading p99 against p50, and why a dead dependency should fail fast rather than hang every request waiting on it.

What it adds up to

Answering one request was the start, and surviving real traffic was the job. The service you can now run is not just a model wrapped in a route; it loads once, refuses input it should not trust, returns predictions aligned to what was asked, and fails in a bounded way under load. That is the difference between a demo that scores a record and a service another team can depend on without you watching it.

What comes next

The endpoint returns a prediction, but a prediction is not yet a product. The next step is the layer a person actually sees: a frontend that calls this endpoint, shows the result, and handles the cases where the model is slow, wrong, or unavailable — turning a number behind an API into something a user can act on. The serving endpoint you built here is exactly what that product calls.