Get the Service Live on the Internet

I deployed a scoring service for the first time and the platform told me it succeeded. The dashboard went green, the revision said healthy, the logs showed the process booting and the model loading. I sent the URL to a colleague to try, and the request hung until it timed out. Nothing in any log said why. I spent the better part of an afternoon reading the application code looking for the bug, and there was no bug in the code. The server was binding to 127.0.0.1 inside its container, so it answered itself perfectly and was invisible to everything in front of it. The deploy had genuinely succeeded at every layer I could see, and the one layer I could not see was the only one that mattered.

Module 6 built the /predict endpoint: a FastAPI service, run by uvicorn, packaged into a Docker image FROM the M5 locked base, loading its serialized model artifact once at startup. Until now that image has only run two places, a laptop and CI, both controlled machines. This lesson is the step where it leaves the building: the same image, unchanged, running on a machine that belongs to someone else, answering a stranger who types a URL. That is the entire goal of this first lesson. The four lessons after it harden what ships here; this one gets a Lending Club default score to come back over the wire on the happy path.

What a deployment actually is, and what one looks like

A deployment looks like one command. A deploy runs, the platform churns for a minute, it prints a URL and the word “healthy,” and the reasonable mental model is that the job is done: the thing is live because the platform said so. That model is wrong in a specific and expensive way, and the way it is wrong is the entire reason this section draws a picture before running anything.

The non-obvious fact is that “the deploy succeeded” and “nobody can reach it” are routinely both true at the same moment. Here is the binding mistake that produces exactly that, the one from the opening incident:

# inside the container: what the M6 server does with a copied laptop default
import uvicorn
from app import app  # the FastAPI app

# WRONG: binds the loopback interface only
uvicorn.run(app, host="127.0.0.1", port=8080)
# the process boots, the model loads, the logs look perfect,
# and the only client that can connect is the container talking to itself

A process bound to 127.0.0.1 listens on the loopback interface, the address a machine uses to talk to itself. The container’s own startup probe runs inside the container, so it connects to 127.0.0.1 and the check passes; the platform sees a passing probe and marks the revision healthy. The router that forwards outside traffic lives in a different network namespace and reaches the container over its real interface, which the server never bound. So the probe is green and the public URL hangs, and no log line reports a failure because, from the process’s point of view, nothing failed. The fix is to bind 0.0.0.0, every interface, on the port the platform assigns.

A deployment is not a new artifact. It is the exact process from before, moved off the developer machine onto an always-on machine with a public address and something in front of it that routes incoming requests. The reason to fix the shape first is that this whole module edits exactly one piece of this shape per lesson, so a stable picture of where the process runs and how the address finds it has to come before any later step can say “this is the hop we are hardening.” Five pieces carry the request: the image (the M6 container, the unit that actually moves), the registry (the image store the host pulls from), the host (the always-on machine the image runs on), the router (the platform front end that reads a request’s path and forwards it to the right container), and the URL (the public address https://host/path a stranger types).

A request reaching /predict is a chain of four hops, and a deployment is the act of making every hop exist for a stranger. Trace them in order:

The unit that moves is the image, not the source. The host does not receive source code; it pulls the exact M6 image from the registry and runs it, so nothing inside the image changes between laptop and host. A registry push stores an image and a pull retrieves it; the registry is the interchange point between the build side and the run side. What changes around the image is everything else: the host has a public IP, an always-on lifecycle, and a router in front. This is why “works on my laptop” and “a customer can use it” are different claims that differ in the host and the router, not in the artifact.

A serverless or scale-to-zero container platform is a host that runs a container and serves it at a URL with no servers to provision or manage: hand it an image, it serves requests. This module uses one deliberately because it collapses the host and the router into a single hand-off: there is no virtual machine to size, no load balancer to wire by hand. The cost and cold-start behaviour of that convenience are real and non-obvious, and they are exactly what Lesson 5 takes apart; for now the property that matters is “give it the image, get a URL.”

With the shape fixed, the smallest real version is four commands: tag the M6 image for the registry, push it, deploy it, then curl the assigned URL from outside. Watch that the deploy step never receives source; it points the compute service at an image the registry already holds.

The exact commands differ by platform, but the three moves are the same on every scale-to-zero host — tag the image for a registry, push it, then point the platform at the pushed image (never at source). In platform-neutral form:

# 1. tag the M6 image for the registry the host will pull from
docker tag lending-predict:m6 <REGISTRY>/lending-predict:m6

# 2. push the image — the registry is the handoff between build and run
docker push <REGISTRY>/lending-predict:m6

# 3. deploy: point the scale-to-zero host at the pushed image (not at source)
#    e.g. `gcloud run deploy`, `flyctl deploy --image`, `render deploy`, etc. —
#    each takes the image reference, a port, and a region; none takes source here.
<PLATFORM-DEPLOY> --image <REGISTRY>/lending-predict:m6 --port 8080 --region <REGION>

# prints a host-assigned URL, e.g.  https://lending-predict-<id>.<platform-domain>

The deploy printed a host-assigned URL: an address the platform chose, HTTP, no custom domain yet. That is the deliberate scope of this lesson; the custom domain and TLS arrive in Lesson 3. Now prove a request can originate off the developer machine and reach the container by hitting the health route from anywhere:

# from a phone, a second laptop, an incognito session — anywhere but the host
curl https://<host-assigned-url>/health
# {"status":"ok"}

That {"status":"ok"} is the reveal: a request left a device on a different network, resolved the name, crossed the connection, passed through the router, and reached the M6 uvicorn process, and nothing about the image changed to make that true. The artifact is byte-identical to what ran in CI; only the host and the router are new.

One trap is worth naming before celebrating, because it is the version of the opening incident still reachable here. If the image listens on a hardcoded port instead of the platform’s injected PORT, the deploy can still report healthy when the numbers happen to line up, then break the instant the platform changes the injected port on a later revision. The symptom is the same hung request from a green deploy, and the root cause is the same violated assumption: the container decided the port instead of the platform. Bind 0.0.0.0 on the platform-injected PORT and the router always has somewhere to forward.

"""Try It: a green deploy with a hung curl from outside.

Predict which hop in the four-hop chain fails first, and identify the one binding
mistake that produces a green deploy with an unreachable URL. Fill in the function
to return the hop number and the corrected bind address.
"""


def diagnose_hung_deploy() -> dict[str, object]:
    # A green deploy + a hung curl from outside.
    # Which hop (1-4) is the first that the request cannot complete?
    # What address must the container bind so the router can forward to it?
    first_failing_hop = 0  # replace with 1, 2, 3, or 4
    correct_bind_address = "?"  # replace with the address to bind
    return {"hop": first_failing_hop, "bind": correct_bind_address}


if __name__ == "__main__":
    print(diagnose_hung_deploy())

"""Solution: diagnose a green deploy with a hung external request."""


def diagnose_hung_deploy() -> dict[str, object]:
    # The deploy is green because the in-container startup probe connects over
    # 127.0.0.1 and succeeds. The request from outside dies at hop 3: the router
    # forwards to the container's real interface, which a 127.0.0.1 bind never
    # opened. Binding 0.0.0.0 on the platform PORT makes hop 3 succeed.
    first_failing_hop = 3
    correct_bind_address = "0.0.0.0"
    return {"hop": first_failing_hop, "bind": correct_bind_address}


if __name__ == "__main__":
    result = diagnose_hung_deploy()
    print(f"first failing hop: {result['hop']}")
    print(f"bind address:      {result['bind']}")
    print("loopback answers the probe; the router reaches a different interface")

A real request hits /predict, not just /health

The health route is now green from a second device, and the temptation is to call the deploy finished: a stranger reached the service, the platform agrees it is healthy, ship it. That conclusion is where deploys go to die quietly, because a passing /health and a working service are not the same claim, and the gap between them is precisely where a deploy succeeds while every prediction returns 500.

The principle is that a shallow liveness check and a real unit of work exercise different amounts of the system. A health probe answers one question, “is the process accepting connections,” which is liveness. It says nothing about readiness: whether the process can actually do its job. A process can be alive and not ready, and the platform’s “deploy succeeded” is driven by the liveness probe, so it marks the revision healthy on a signal that touches none of the work the service exists to do. This is the canonical liveness-versus-readiness gap, and it is the one this module grounds here and back-references later: Lesson 2 shows why it turns into an outage under a single environment, and Lesson 4 builds the readiness probe that finally closes it. Watch a deliberately broken build pass the exact check the platform trusts:

# a build with a wrong model path or an unreachable registry
@app.get("/health")
def health() -> dict[str, str]:
    return {"status": "ok"}        # touches nothing: no model, no features, no schema

@app.post("/predict")
def predict(loan: LoanRecord) -> dict[str, float]:
    score = MODEL.predict_proba(...)   # MODEL is None — load failed at startup
    return {"probability": float(score)}
# GET  /health  -> 200 {"status":"ok"}   <- platform marks the deploy healthy
# POST /predict -> 500 AttributeError    <- every real request fails

/health returns a hardcoded literal; it never references the model, the feature code, or the request schema, so it cannot detect that any of them are broken. Three failures hide behind it specifically: a wrong model path, a missing env var the loader reads, or a registry the host cannot reach. All three fail at model-load or first-predict and never at /health, so the platform ships a broken /predict behind a green deploy. The deploy is not real until a Lending Club record goes in over the public URL and a score comes back; that is the first moment the deployment does the job M6 built it for.

The reason /predict is the honest check is that it exercises the full chain the M6 build put in place. uvicorn hands the request body to FastAPI; FastAPI runs the M6 pydantic request model over the JSON, the same boundary contract from the Python module, now validating on a machine the developer does not own; the validated record flows through the feature transform; and the loaded model produces a score. A POST request carries a body of data to the server (here, the loan record), unlike a GET that only retrieves a resource. FastAPI performs schema validation at the boundary: the pydantic model checks the JSON before any handler code runs, so a malformed record is rejected at the door rather than corrupting a prediction. The model itself loaded once at process start: M6 reads the serialized artifact into memory before serving the first request, and prediction latency, the time from request to answer, is separate from that one-time load (the cold-start cost of the load is Lesson 5’s subject).

A successful /predict therefore proves four things /health cannot: the artifact was reachable and deserialized, the feature code ran, the request schema matched, and the score is in range. The diagram below makes the gap concrete: the same record scored locally and live land identically, while a broken build shows green health beside a 500 on predict.

[POST /predict laptop -> 0.182] as local
[POST /predict live url -> 0.182] as live
[GET /health -> 200 ok] as health
[POST /predict broken -> 500] as broken
local --> live : same image, same record, identical score
health --> broken : health green while predict 500s

The two columns of that diagram are the whole lesson in one frame. On the left, the same Lending Club record scores 0.182 on a laptop and 0.182 at the live URL: behavioural identity, where the deploy changed reachability, not behaviour. A byte-identical artifact running the same input produces the same output; only the host and router differ. On the right, a broken build returns green health and a 500 on predict at the same instant, the gap a liveness probe cannot see. Confirm the left column on a real deploy by sending one record over the wire:

# POST a Lending Club record to the live /predict — the body carries the record
curl -X POST https://<host-assigned-url>/predict \
  -H "Content-Type: application/json" \
  -d '{"loan_amnt": 10000, "term": 36, "int_rate": 13.5, "annual_inc": 65000,
       "dti": 18.2, "fico_range_low": 690, "revol_util": 42.0}'
# {"probability": 0.182}     <- identical to the M6 local result for this record

If that probability matches what M6 returned locally for the same record, the deploy is real and honest: reachability changed, behaviour did not. If the score differs, the deploy changed something it should not have, almost always config rather than the image, which is exactly why Lesson 2 promotes config separately from the artifact. A matching score is the first moment this service has done, for a stranger, the job it was built to do.

"""Try It: decide whether a revision is actually serving.

Given a health result and a predict result for the same record that scored 0.182
locally, fill in the logic that distinguishes "alive" from "ready and correct."
"""


def revision_is_serving(
    health_status: str,
    predict_status: int,
    predict_score: float | None,
    local_score: float,
) -> str:
    # health_status: "ok" if /health returned 200
    # predict_status: HTTP status from POST /predict (200, 500, ...)
    # predict_score: the probability returned, or None on error
    # local_score: what M6 returned locally for this same record (0.182)
    # Return one of: "alive but broken", "reachable but wrong", "serving correctly"
    return "?"  # replace with the decision


if __name__ == "__main__":
    print(revision_is_serving("ok", 500, None, 0.182))
    print(revision_is_serving("ok", 200, 0.182, 0.182))
    print(revision_is_serving("ok", 200, 0.301, 0.182))

"""Solution: distinguish alive from ready-and-correct for a revision."""


def revision_is_serving(
    health_status: str,
    predict_status: int,
    predict_score: float | None,
    local_score: float,
) -> str:
    # /health green only proves liveness -- the process accepts connections.
    if predict_status != 200 or predict_score is None:
        return "alive but broken"  # green health hides a 500 on predict
    # Reachable, but readiness includes correctness: behavioural identity.
    if abs(predict_score - local_score) > 1e-9:
        return "reachable but wrong"  # config drifted: deploy changed behaviour
    return "serving correctly"  # reachability changed, behaviour did not


if __name__ == "__main__":
    print(revision_is_serving("ok", 500, None, 0.182))  # alive but broken
    print(revision_is_serving("ok", 200, 0.182, 0.182))  # serving correctly
    print(revision_is_serving("ok", 200, 0.301, 0.182))  # reachable but wrong

Summary

• A deployment is the exact M6 image, unchanged, moved onto an always-on host with a public address and a router in front. Five pieces carry it: image, registry, host, router, URL. The image is the only thing that moves; everything else is new context around it.
• A request reaches /predict through four hops: DNS resolves the name, TCP/TLS opens the connection, the platform router reads the path and forwards, and uvicorn/FastAPI dispatch. Each later lesson in this module edits exactly one hop.
• The container must bind 0.0.0.0 on the platform-injected PORT, never 127.0.0.1. The loopback bind passes the in-container probe and turns green while the external router has nothing to forward to: a healthy deploy that no one can reach.
• A green /health proves liveness (the process accepts connections), not readiness (it can do its job). A wrong model path, a missing env var, or an unreachable registry all pass /health and 500 on /predict.
• The deploy is real only when a Lending Club record scores the same over the wire as it did locally: behavioural identity. The deploy changes reachability, not behaviour; a different score means config drifted, not the image.

Check your understanding:

• In one sentence each: what is an image, a host, and a URL, and which of the three is the thing a stranger actually types?
• A deploy reports success but curl from another machine hangs. Which hop in the four-hop chain is the first place to look, and what one binding mistake produces exactly that symptom?
• Without looking back: name two failures that pass /health and fail /predict, and state why the platform still reports the deploy as succeeded.
• The same record scores 0.182 locally and 0.301 over the wire from the same image. What changed, and which lesson fixes the thing that changed?