Module 6

APIs & Model Serving

Turn a model into a service other code can call: schema-validated, batch-capable, honest about its latency, and a stable contract for its consumers.

Instructor Sarah Floris · Lead ML Engineer Coming soon

Included with Pro

Coming soon Not yet available

Requires Packaging & Reproducibility

What you will learn

Serve a model under real traffic

Build a /predict endpoint that loads the model once
Keep model loading off the critical request path
Validate and reject malformed input at the boundary
Add a batch path and measure latency
Survive real traffic — tail latency, degradation, scale

Prerequisites

A locked, reproducible Docker image

You arrive with a Docker image that builds reproducibly and runs the model with config and secrets kept safe. This module deploys that image as a long-running FastAPI service and hardens it against the boundary cases real traffic creates.

Chapter list

7 chapters

7 lessons

01	What This Module Covers The arc of APIs & Model Serving: a working /predict endpoint, model loading that survives traffic, input the endpoint refuses to trust, a batch path with a latency budget, and an endpoint that degrades gracefully under real load.
02	Get a Working /predict Endpoint Running Stand up a real FastAPI service that loads the Lending Club model once at startup, takes a request, and returns a prediction you can curl. The endpoint every later lesson hardens.
03	Make Model Loading Survive Real Traffic Harden the part of the endpoint that breaks first under load — where and when the model is loaded — and containerize it on the locked training image so it serves in the same environment it was trained in
04	Make the Endpoint Reject Bad Input Reject malformed input at the boundary with pydantic schemas, and exclude the leakage features from the request contract.
05	Add a Batch Path and Measure What It Costs Build a batch route that scores many records in one call, keep its output index-aligned with its input, and measure end-to-end latency at the boundary instead of timing model.predict() and guessing the rest.
06	Survive Real Traffic: Tail Latency, Degradation, Scale The last hardening pass: why p99 diverges from p50 under load, what in-flight requests do when the model dies, and where a Python endpoint stops scaling.
07	What You Built in This Module A recap of APIs & Model Serving: a /predict endpoint that loads once and survives traffic, refuses bad and leaky input, returns aligned batch predictions, and degrades gracefully under load — and where the system goes next.

Ready to start?

This module is coming soon. Join the waitlist to be notified.

Coming soon Create account