Make It Degrade Gracefully

I built this dashboard myself — the same fetch → render page from the last lesson — and demoed it to our credit lead so she could check default risk on applications without bugging me. The audience was the underwriting team, a small group, on a handful of requests a minute. It was low-traffic enough that I never thought failure handling mattered. The page called /predict, got a probability, rendered it, clean. Then I pushed a model update, the serving container restarted, the endpoint returned a 503 while it came back, and the page did three wrong things at once: it threw an uncaught error in the console, it left the last prediction sitting on screen as if it were fresh, and it gave her no way to know anything was wrong. She acted on a stale number, and I only found out when she asked why the same applicant kept scoring identically.

Working on the happy path is not the same as safe when the backend blinks. In the last lesson you wired one click through fetch → await response.json() → render, and every step assumed the response arrived and carried a prediction. That assumption is the gap. The endpoint you call is the M6 /predict service, and a service deploys, overloads, and restarts — and on a two-or-three-person team, the backend is you, mid-deploy. This lesson goes back to that working dashboard and hardens the failure path: detect that a call did not produce a prediction, then degrade to an honest “unavailable” state instead of blanking out or leaving a stale guess on screen. The two halves are one rule. You cannot degrade honestly until you can detect failure, so detection comes first.

I built this dashboard myself — the same fetch → render page from the last lesson — and demoed it to our credit lead so she could check default risk on applications without bugging me. The audience was the underwriting team, a small group, on a handful of requests a minute. It was low-traffic enough that I never thought failure handling mattered. The page called /predict, got a probability, rendered it, clean. Then I pushed a model update, the serving container restarted, the endpoint returned a 503 while it came back, and the page did three wrong things at once: it threw an uncaught error in the console, it left the last prediction sitting on screen as if it were fresh, and it gave her no way to know anything was wrong. She acted on a stale number, and I only found out when she asked why the same applicant kept scoring identically.

Working on the happy path is not the same as safe when the backend blinks. In the last lesson you wired one click through fetch → await response.json() → render, and every step assumed the response arrived and carried a prediction. That assumption is the gap. The endpoint you call is the M6 /predict service, and a service deploys, overloads, and restarts — and on a two-or-three-person team, the backend is you, mid-deploy. This lesson goes back to that working dashboard and hardens the failure path: detect that a call did not produce a prediction, then degrade to an honest “unavailable” state instead of blanking out or leaving a stale guess on screen. The two halves are one rule. You cannot degrade honestly until you can detect failure, so detection comes first.

A response is not a success: check the status first

The mental model that ships this bug is that fetch throws when the server fails. A competent engineer reasons: a network call either works or it raises, so wrapping it in a try/catch and then reading the body in the success path must be safe — if the server had returned 503, control would have jumped to the error handler. That model is wrong in a way that stays invisible until a real 503 arrives, because fetch resolves on a 503. The success path runs. The body gets parsed. And what gets parsed is not a prediction.

Here is the wrong model running against a 503. The handler catches transport errors and otherwise trusts the body, exactly as the throw-on-failure model predicts.

// WRONG: assumes a resolved fetch means a successful request
async function getPrediction(record) {
  try {
    const response = await fetch(API_URL, {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify(record),
    });
    const data = await response.json();   // runs even on 503
    render(data.probability);             // renders undefined, or throws
  } catch (err) {
    showError();                          // only fires on a NETWORK failure
  }
}
// Server returns 503 with an HTML error page from the load balancer.
// fetch RESOLVES. response.json() tries to parse "<html>...</html>" as JSON.
// Either it throws a confusing SyntaxError, or (empty body) data.probability
// is undefined and renders as a blank or "undefined" on the page.

The reason fetch does this surprises everyone the first time, and the cause is where it draws its line. The browser’s fetch rejects its promise only when the request fails to complete the HTTP exchange at all — DNS that will not resolve, a connection refused, the network dropping mid-flight, a request blocked by the cross-origin policy. From the browser’s point of view, a server that answered 503 Service Unavailable did exactly what HTTP asks of it: it completed a valid request/response exchange. MDN states this directly — a fetch() promise “does not reject if the server responds with HTTP status codes that indicate errors (404, 504, etc.). Instead, a then() handler must check the Response.ok and/or Response.status properties.” The status code is application-level information riding inside a transport-level exchange that succeeded, so fetch resolves and hands you a Response object whose .ok is false and whose .status is 503. The try/catch was guarding the wrong layer.

The single signal that separates the two cases is Response.ok, which is true only for a status in the 200–299 range. That is the one property distinguishing “the server returned a prediction” from “the server told me it failed,” and nothing in the language forces you to read it. The correct handler checks the status before it touches the body, because the body is only trustworthy if the status says so.

// RIGHT: check the status before reading the body
async function getPrediction(record) {
  const response = await fetch(API_URL, {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify(record),
  });
  if (!response.ok) {
    return { ok: false, status: response.status };  // never touch the body
  }
  const data = await response.json();
  return { ok: true, prediction: data };
}

The page JavaScript ships to a browser running on a machine and a network you do not control, talking to a server that can be mid-deploy or overloaded. That is the deeper truth under the rule: the browser is an untrusted client, so every happy-path assumption — “the fetch returned, therefore I have a prediction” — is one the environment is free to violate. To show the routing without a browser, model it in Python the way an httpx client mirrors the same request/response sequence. The function below receives a fake response object, then branches on the status the way the corrected handler does. Watch the same kind of resolved object route to two opposite outcomes depending only on the status.

python
import json
from dataclasses import dataclass


@dataclass
class FakeResponse:
    status: int
    body: str  # raw bytes the server sent: JSON on success, an HTML error page on 503

    @property
    def ok(self) -> bool:
        # mirrors the browser's Response.ok: true only for 200-299
        return 200 <= self.status < 300


def handle(response: FakeResponse) -> dict:
    # The status check happens BEFORE the body is read.
    if not response.ok:
        # error path: never parse the body, route to an error state
        return {"state": "error", "status": response.status}
    # success path: only here is the body trusted to be a prediction
    data = json.loads(response.body)
    return {"state": "success", "probability": data["probability"]}


success = FakeResponse(status=200, body='{"probability": 0.31, "label": "repay"}')
failure = FakeResponse(
    status=503, body="<html><body>503 Service Unavailable</body></html>"
)

print("resolved, ok =", success.ok, "->", handle(success))
print("resolved, ok =", failure.ok, "->", handle(failure))

Both responses resolved — neither raised — yet one is a prediction and one is a service failure, and only .ok told them apart. The error branch never calls json.loads, which is the whole point: parsing the 503’s HTML body as JSON is the failure mode. This is the same class of bug as a downstream batch job that parsed a load balancer’s 503 HTML page as JSON and wrote null scores into a table — the client side has the identical failure with a worse blast radius, because a human reads the result and acts on it.

The named failure mode here is parse-the-error-body: the symptom is either a confusing SyntaxError in the console (the HTML error page is not valid JSON) or, when the error body is empty, data.probability is undefined and renders as a blank or the literal string undefined on the page. The root cause is reading the body before checking the status; the boundary it violates is the transport-versus-application split — fetch reports only transport failure, so application failure must be read off the status. The non-obvious cost of the throw-on-failure mental model is that the bug is invisible in every test you run by hand, because you test against a server that is up. It only appears when a real 503 arrives in production, which is exactly the moment no one is watching the console.


Try It 1

A handler calls .json() unconditionally — the resolved response is trusted no matter what status it carries. Below is a Python model of it receiving a 503. Predict what handle_broken does before running it, then fix handle_fixed: insert the status check so a 503 routes to an error state instead of a parse attempt.

python
import json
from dataclasses import dataclass


@dataclass
class FakeResponse:
    status: int
    body: str

    @property
    def ok(self) -> bool:
        return 200 <= self.status < 300


def handle_broken(response: FakeResponse) -> dict:
    # BUG: reads the body with no status check
    data = json.loads(response.body)  # blows up on a 503 HTML body
    return {"state": "success", "probability": data["probability"]}


def handle_fixed(response: FakeResponse) -> dict:
    # TODO: check response.ok BEFORE reading the body; return an error state on failure
    return {"state": "unknown"}  # placeholder -- replace


failure = FakeResponse(status=503, body="<html>503 Service Unavailable</html>")
print(handle_fixed(failure))
Hint Read the section's rule literally: the body is only trustworthy if the status says so. What is the one property that is `true` only for a 200–299 status? State your hypothesis about which line in `handle_broken` raises on a `503`, then guard that line so it never runs unless the status is in the success range. Re-read "A response is not a success."

Solution

The fix checks response.ok first and returns an error state carrying the status, so the body is never parsed on a failure. Watch the 503 route to an error state instead of raising.

python
import json
from dataclasses import dataclass


@dataclass
class FakeResponse:
    status: int
    body: str

    @property
    def ok(self) -> bool:
        return 200 <= self.status < 300


def handle_fixed(response: FakeResponse) -> dict:
    if not response.ok:
        return {"state": "error", "status": response.status}
    data = json.loads(response.body)
    return {"state": "success", "probability": data["probability"]}


failure = FakeResponse(status=503, body="<html>503 Service Unavailable</html>")
success = FakeResponse(status=200, body='{"probability": 0.31}')
print(handle_fixed(failure))
print(handle_fixed(success))

The error path returns {"state": "error", "status": 503} and never touches the body, so the SyntaxError from parsing an HTML page is gone. The success path is unchanged. The status check did not replace the error handling — it added the layer fetch does not report, and that layer is where every application failure lives.

The scrolly below walks one request after the fetch promise has resolved, so the resolved object’s two opposite meanings — and the single check that separates them — are visible in order.

The promise resolves. The page now holds a Response object. At this single point in time, a successful prediction and a 503 service failure look identical — both are resolved responses. Nothing has distinguished them yet.

The wrong path: read the body immediately. The throw-on-failure model assumes "resolved means success," so it goes straight to response.json(). No status was checked, because the model believed a failure would have rejected the promise.

It parses an error page. The 503 body is an HTML page from the load balancer, or empty. json() either throws a SyntaxError or yields an object with no probability, which renders as a blank or undefined on the page. The failure is silent or confusing.

The right path: check the status first. Before touching the body, read response.ok (true only for 200–299) or response.status. This is the layer fetch does not report by rejecting — the application-level signal riding inside the completed transport exchange.

Branch on the status. A 2xx parses probability and renders it. An error status never touches the body and routes to an error state. The same resolved object meant two opposite things; the status check is the only thing that told them apart.

The reveal is the order: resolve, then check, then branch. Detection is now in place — the handler can tell a prediction from a service failure. What it does with that failure is the next half of the lesson, because routing a 503 to an “error state” is only honest if that state changes what the user sees.

Graceful degradation: a useful page when the model is unavailable

The mental model that ships the second half of the incident is that detecting the failure is enough — once the handler routes a 503 to the error branch, the dangerous case is handled. The reasoning is reasonable: the error path runs, so the bug is caught. But “caught” and “communicated” are different things, and the gap between them is where the stale number lives. An error branch that catches the failure and then does nothing visible leaves the previous prediction on screen, and the user cannot tell the difference.

Here is the error branch that “handles” the failure by logging it. The detection from the last section is wired in correctly. Watch what stays on screen.

async function onPredict(record) {
  const result = await getPrediction(record);   // returns {ok, status} on failure
  if (!result.ok) {
    console.error("predict failed:", result.status);  // "handled" — and silent
    return;   // <- the previous prediction is STILL on the page
  }
  render(result.prediction);
}
// The 503 was detected and logged. But #result still shows "0.41 — likely to
// repay" from the LAST successful call. The user sees a fresh-looking number
// the model never produced for THIS applicant.

A rendered value has no timestamp and no liveness signal — once written to the page it persists until something overwrites it. That is a DOM fact, not a configuration: the text in #result does not clear itself when a later fetch fails. So if the failure branch does nothing, the previous prediction stays on screen, visually identical to a fresh one, and the user cannot distinguish “the model says 0.41 for this applicant” from “the model is down and this is from two minutes ago.” This is the eventual-consistency problem the UI owns: any information is potentially outdated when it is displayed, and the interface is what must keep a user from acting on a stale view as if it were current. The failure branch is where that responsibility is exercised.

When the prediction cannot be obtained, the dashboard has exactly three options, and only one is acceptable. The decision is which state to render on failure.

Show stale data

When: never — it is the accidental default of doing nothing on the error branch. Failure modes: the last prediction persists with no liveness signal, and the user acts on a number the model never produced for the current input. This is the most dangerous option because it looks correct. There is no error, no blank, no symptom — only a wrong answer presented as a right one. It was the cause of the opening incident: a 503 left the last score on screen and the credit lead acted on it.

Blank the page

When: never as the whole strategy — clearing the result without explanation looks like the app crashed. Failure modes: the user cannot tell “broken” from “still loading” from “model down,” loses trust in the tool, and refreshes into the same blank. Blanking removes the stale lie but replaces it with silence, which the user reads as a defect in the page rather than a transient backend failure.

Explicit unavailable state

When: always, on any failed or non-ok prediction call. Failure modes: the only way this fails is forgetting to also disable the act-on-it control — leaving a live button the user clicks expecting a result, or an enabled “approve” action next to a prediction that no longer exists. The state must replace the stale value and disable the dependent action.

The judgment call specific to ML is what to degrade to. A numeric field can fall back to a last-known value or a zero; an ML prediction cannot. There is no “neutral” probability to show — every value in [0, 1] is a claim about this specific input, and the model made none. The reliable-ML guidance is to have a defined fallback behaviour when a model fails, not a guessed output; a fabricated score is exactly the bad output a fallback is supposed to prevent. So the only honest fallback is “no prediction right now,” rendered as an explicit unavailable state, never a manufactured number.

The page is a small state machine, and graceful degradation is the requirement that the error state actively replaces whatever was on screen. The Python model below runs the page through its states and prints the user-visible string for each branch. Watch that the failure branch returns a different visible string and a disabled-action flag — not the previous value.

python
from dataclasses import dataclass


@dataclass
class PageState:
    status: str  # "loading" | "success" | "error"
    message: str  # exactly what the user reads
    action_enabled: bool  # is the act-on-it control clickable?


def reduce(prev: PageState, event: str, payload: dict | None = None) -> PageState:
    if event == "click":
        return PageState("loading", "Scoring...", action_enabled=False)
    if event == "resolved_ok":
        p = payload["probability"]
        return PageState(
            "success", "Prediction: " + format(p, ".2f"), action_enabled=True
        )
    if event == "resolved_error":
        # the failure branch: REPLACE the stale value, DISABLE the action
        return PageState(
            "error",
            "Prediction unavailable - the model is not responding. Try again.",
            action_enabled=False,
        )
    return prev


state = PageState("success", "Prediction: 0.41", action_enabled=True)  # last good value
print("before failure:", state.message)

# the 503 arrives -- the failure event fires
state = reduce(state, "resolved_error")
print("after failure: ", state.message, "| action_enabled =", state.action_enabled)

The before line is the stale 0.41 from the previous success. After the failure event, the message is replaced with an explicit “unavailable” string and the action is disabled, so the absence of a prediction is visible rather than inferred. The fix was not a retry loop and not a fancier error — it was making the failure overwrite the screen and removing the control that depended on a fresh prediction.

The named failure mode is the silent-stale render: the symptom is a prediction that looks fresh but is from an earlier input; the root cause is a failure branch that detects the error but does not write to the DOM; the boundary it violates is the one the UI is responsible for — never presenting stale query data as current. The non-obvious cost of the obvious fix is that you must remember to disable the dependent control as well as replace the text. Replacing only the message leaves a dead “approve” or “predict-and-act” button next to an unavailable state, and a user who clicks it acts on nothing — a quieter version of the same bug, where the action ran against a prediction that does not exist.

The diagram below draws the page as its states and the legal transitions between them. The bug is a missing transition: the error path that does nothing, so the success state silently persists.

[idle] as idle
[loading] as loading
[success\n(prediction shown,\naction enabled)] as success
[error\n(unavailable shown,\naction DISABLED)] as error

idle --> loading : click
loading --> success : response.ok
loading --> error : non-ok / network fail
success --> loading : next click
error --> loading : next click

The whole point is that loading → error is a real transition that replaces the screen, not a no-op that leaves success showing. Drawing the states explicitly makes the bug visible as the transition that does nothing — the failure path that logs and returns, leaving the last success render in place. The status check from the first section is what fires the loading → error edge; degradation is what that edge does.


Try It 2

The reducer below has a failure branch that “handles” the error by returning the previous state unchanged — the silent-stale bug. For the dashboard, write the error branch so it replaces any stale prediction with an explicit “prediction unavailable” message and disables the action button.

python
from dataclasses import dataclass


@dataclass
class PageState:
    status: str
    message: str
    action_enabled: bool


def reduce(prev: PageState, event: str, payload: dict | None = None) -> PageState:
    if event == "click":
        return PageState("loading", "Scoring...", action_enabled=False)
    if event == "resolved_ok":
        p = payload["probability"]
        return PageState(
            "success", "Prediction: " + format(p, ".2f"), action_enabled=True
        )
    if event == "resolved_error":
        # BUG: does nothing -- the stale success state persists
        return prev  # TODO: replace the screen and disable the action
    return prev


state = PageState("success", "Prediction: 0.41", action_enabled=True)
state = reduce(state, "resolved_error")
print(state.message, "| action_enabled =", state.action_enabled)
Hint What does "graceful degradation" require the failure branch to do beyond detecting the error — re-read the three-option decision. The error branch must produce a state whose message the user cannot mistake for a fresh prediction, and whose action flag does not invite a click that acts on nothing. Do not return `prev`. There is no safe number to show, so the message names the unavailability, not a value.

Solution

The error branch builds a new state with an explicit unavailable message and a disabled action, so the stale 0.41 is gone and the button cannot be clicked into a missing prediction.

python
from dataclasses import dataclass


@dataclass
class PageState:
    status: str
    message: str
    action_enabled: bool


def reduce(prev: PageState, event: str, payload: dict | None = None) -> PageState:
    if event == "click":
        return PageState("loading", "Scoring...", action_enabled=False)
    if event == "resolved_ok":
        p = payload["probability"]
        return PageState(
            "success", "Prediction: " + format(p, ".2f"), action_enabled=True
        )
    if event == "resolved_error":
        return PageState(
            "error",
            "Prediction unavailable - try again in a moment.",
            action_enabled=False,
        )
    return prev


state = PageState("success", "Prediction: 0.41", action_enabled=True)
state = reduce(state, "resolved_error")
print(state.message, "| action_enabled =", state.action_enabled)

The stale 0.41 is overwritten and the action is disabled, so a user cannot read an old score as current or click into a prediction that was never produced. The error state carries no fabricated number, because there is no honest default for a probability — “unavailable” is the only true thing the page can say when the model did not answer.

Summary

  • fetch rejects its promise only on a transport failure (DNS, connection refused, dropped network) — a 4xx or 5xx resolves successfully, so the handler must read response.ok (true only for 200–299) before touching the body.
  • The named failure is parse-the-error-body: calling .json() on a 503 parses an HTML error page or empty body, throwing a SyntaxError or rendering undefined as a prediction.
  • A rendered value persists until overwritten — it has no liveness signal — so a failure branch that does nothing leaves the previous prediction on screen, looking fresh. That silent-stale render is the most dangerous of the three failure options.
  • On failure the page has three choices — show stale (lies), blank (looks crashed), or explicit unavailable (honest) — and only the third is acceptable; it must replace the screen and disable the act-on-it control.
  • An ML prediction has no safe default value: there is no neutral probability, so the only honest fallback is “no prediction right now,” never a fabricated number.

Check your understanding:

  • Does fetch throw when the server returns 503? If not, what one property tells you the request actually failed, and at what point must you read it?
  • When a prediction call fails, why is showing the last prediction more dangerous than blanking the page?
  • What is the only honest fallback for an ML prediction specifically, and why can it not be a number?
  • Without looking back: the error branch replaces the message but forgets one thing, and the user clicks into a prediction that does not exist — what did it forget?

This lesson is part of Pro

The Ship a Machine Learning Product path — every lesson, capstone, and the failure modes free tutorials skip. Sign in if you already have Pro, or unlock it below.

Unlock with Pro Sign in