Make It Slim

I once packaged a churn scorer, the one that had to chew through hundreds of millions of daily event rows across a dataset of about thirty gigabytes, so it carried the full numeric stack, into an image where “it works on my machine” was literally true and completely useless. The image was multi-GB. It took the CI registry minutes to push and minutes more to pull on every deploy, and it carried a full build toolchain, the package-manager cache, and a compiler the running container never touched once. I shrank a production image like that from multi-GB to a few hundred MB without changing a line of model code; the entire difference was what got left out and which layers were ordered to cache. A working image that nobody can ship fast enough loses every deploy race, and the team that owns it spends its day watching a progress bar instead of shipping.

In the last lesson you took the reproducible image from Lesson 2 and stopped it from shipping its own secrets: config moved to os.environ, secrets injected at run time, and a .dockerignore kept the .env out of the build context. That image now builds the same twice, reads its configuration safely, and carries no credentials. It also weighs multi-GB, because the numeric stack a scorer needs to run pulls in a build toolchain it only needs to install. This lesson is the final hardening pass. It does two things that look unrelated and are the same thing: it diagnoses “works on my machine” as a specific class of systems failure, and it cuts the image down, because both come from the same question, which is what does the running container actually depend on, and what is along for the ride.

“Works on my machine” is a missing-environment failure

“Works on my machine” reads, plausibly, as the other engineer doing something wrong: a stale checkout, a missed step, a fat-fingered command. Treating it as a personality failure is the wrong model, and it is wrong in a way that costs hours, because it sends people looking at the human instead of the gap. “Works on my machine” is a precise, reproducible systems failure: the running environment depends on something present on the author’s machine and absent on the target. A program’s behaviour is not determined by its source alone. It is determined by its full environment: the interpreter version, every installed library version, the OS-level shared libraries those libraries link against, the locale, the files on disk, and the environment variables in scope. Any one of those that exists on the author’s box and not on the target produces a runtime failure the author genuinely cannot reproduce, because on the author’s machine nothing is missing.

Closing that gap is what Docker exists to do. An image packages an application together with its dependencies into one portable artifact, so the same software runs the same way on any machine; the mechanism is that the image carries the environment with the code instead of assuming the target already has it. An image is a pre-prepared root filesystem: the OS files, the interpreter, the installed libraries, frozen into one shippable thing. When that artifact runs somewhere clean and still fails, it has not closed the gap; it has only narrowed which class of gap is left. Naming those classes is what tells you whether the image actually fixed the parity problem or merely relocated it.

Five classes of missing-environment failure exist, and they are not five unrelated bugs. They are five places the same parity gap can open, ordered from the one Docker closes for free to the one it cannot see at all. The first is a missing library: the code imports something the target does not have installed, which the image closes by construction, because installing the dependencies into the image is the whole point. That installed library can still be the wrong version, the second class, present but a different release, which the lock file from the build-reproducibility lesson pins shut. A correct version can still fail to load, which is the third class and the one that escapes both fixes above: a missing OS-level shared library, a system .so that a compiled wheel links against, present on the dev laptop and absent in a stripped base image. Even with every library present and loadable, the code can read a value that lived only in the author’s shell, the fourth class, dev-only configuration, which the previous lesson’s injected environment variables handle. The quietest is last: a file on the author’s disk the code opens at run time, which only a deliberate COPY brings along. Each later class survives the fix for the earlier one, so an image only closes the gap for the classes it actually accounts for.

Class three is the one that surprises people, because it is invisible until the image runs somewhere truly clean. Here is the failure: a numeric library installs as a wheel, a prebuilt binary package, and that binary was compiled against a system shared library, a .so file the operating system provides. A dynamically linked binary does not contain the code of the libraries it uses; the shared .so must be present and loaded at run time, and if it is absent the program fails to start at all. On a full developer machine that .so is almost always already there, dragged in by some other package. In a stripped base image it is not. The model scores fine locally and the container dies on launch with a missing-shared-library error that names a file nobody on the team has heard of.

Reverting to the fat base image is the wrong instinct at that moment, reached for because the fat image “worked.” That trades the whole lesson away to paper over one missing file. The right fix is to install the one system library into the base layer. The cost of the wrong instinct is not only a bigger image; it is that the diagnostic signal is now lost entirely, because a fat base masks which dependency was missing, so the next clean environment surfaces the same class of bug again with no clue attached. The diagnostic for “which .so” exists: ldd lists a binary’s shared-object dependencies and marks the ones it cannot find. This is the section’s named failure mode, a missing OS-level shared library, class three, and its fix is one apt-get install in the base layer, not a 4 GB retreat.

Diagnosing by class instead of by symptom is the cleanest way to internalize the taxonomy. Below, the classifier reduces each failure to one fact, what the host has that the image does not, then maps that fact to its class and its fix, which is the move a staff engineer makes before touching the Dockerfile.

def diagnose(host_has: str, image_lacks: str) -> tuple[str, str]:
    """Map a 'works on my machine' report to its failure class and fix."""
    table: dict[str, tuple[str, str]] = {
        "library": ("1: missing library", "image installs the dependency"),
        "version": ("2: wrong version", "lock file pins the exact version"),
        "shared_lib": (
            "3: missing OS shared lib",
            "apt-get install the .so in the base layer",
        ),
        "config": ("4: dev-only config", "inject it at run time via -e / os.environ"),
        "file": ("5: dev-only file", "COPY the file into the image"),
    }
    return table[image_lacks]


reports: list[tuple[str, str]] = [
    ("numpy 1.26 installed", "library"),
    ("scikit-learn 1.4.2 pinned", "version"),
    ("libgomp.so.1 from the OS", "shared_lib"),
    ("SCORE_THRESHOLD in my shell", "config"),
    ("a local thresholds.json", "file"),
]

for host_has, image_lacks in reports:
    failure_class, fix = diagnose(host_has, image_lacks)
    print(f"host had: {host_has:<28} -> class {failure_class:<26} fix: {fix}")

The point the output makes is that “works on my machine” is never one bug; it is one of five, and the class names the fix before you open the Dockerfile. The shared-library row is the one that escapes the obvious fixes, and it is exactly the class a slim base image is most likely to expose, which is where the next section starts: the base image you choose is both the floor of the image size and the place class-three failures are born.

def classify(report: str) -> tuple[int, str]:
    """Return (failure_class_number, the_fix) for each report."""
    # Report A: "I pinned pandas==2.2 in the lock; the image has pandas 2.1."
    # Report B: "The container dies on launch: libgomp.so.1 not found."
    # Report C: "It only scores correctly when SCORE_THRESHOLD is set; the image has no default."
    a: tuple[int, str] = (0, "TODO")
    b: tuple[int, str] = (0, "TODO")
    c: tuple[int, str] = (0, "TODO")
    return a, b, c


print(classify("placeholder"))

def report_fixes() -> list[tuple[int, str]]:
    """The (failure_class_number, the_fix) for each of the three slim-image reports."""
    return [
        (2, "lock file pins the exact version into the image"),
        (3, "apt-get install the system .so in the base layer"),
        (4, "inject via docker run -e and read with os.environ"),
    ]


for label, (cls, fix) in zip(("A", "B", "C"), report_fixes()):
    print(f"report {label}: class {cls} -> {fix}")

Slim the base and drop what runtime never uses

Image bloat looks, intuitively, like a heavy model and heavy data making a heavy image, so shrinking it would mean shrinking the model. That theory predicts the wrong fix and points at the wrong bytes. Most image bloat is not the model. It is the base image plus build-time tooling the running container never executes: the C compiler, the -dev header packages, and the package-manager cache left behind after an install. A scorer needs gcc to build a numeric wheel that has no prebuilt binary; it never needs gcc to run that wheel. Every byte of build tooling that survives into the shipped image is dead weight that the registry pushes and every deploy pulls, for code that never calls it.

The base image is the floor of the image size, and the three common choices trade size against build risk in a way that is a genuine decision, not a default. A full python image carries a complete build toolchain and OS: easiest to get working, multi-GB to ship. A python:*-slim image is a stripped-down Debian with Python and little else, kept as small as possible. Alpine Linux is smaller still and popular precisely because it is designed to be tiny. The mistake is to read “smaller is better” off that list and reach for Alpine, because the size axis is not the only axis, and on the axis that actually bites an ML image, Alpine is the worst choice.

Here is what the size-only theory predicts versus what happens. Alpine uses musl libc instead of glibc, a different implementation of the C standard library. The prebuilt wheels that numpy, scipy, and scikit-learn ship are manylinux wheels, which target glibc: a manylinux wheel is defined to work on any distribution based on a compatible glibc. Those wheels do not match musl. So on Alpine the package manager cannot use the prebuilt binary and falls back to compiling each numeric package from source, which is why a “smaller” base can produce a slower, riskier build. This is the section’s named failure mode: a team moves to Alpine to shrink an image, every numeric wheel recompiles from source against musl, the build time multiplies, and a wheel that has no clean musl build path fails outright, so the image got smaller and the build broke. The packaging ecosystem treats this as real enough that it ships a separate musllinux wheel tag and separate Alpine-based build images, precisely because musl and the standard glibc wheels are incompatible.

# What "smaller is better" predicts vs. what an Alpine ML build actually does.
FROM python:3.12-alpine          # tiny base — the size-only theory is happy here
RUN pip install scikit-learn     # no musl wheel -> compiles from source
# ... the build now drags in gcc, gfortran, BLAS headers, and minutes of CPU,
#     and if one transitive numeric dep has no clean musl build, the build fails.
# The image you wanted smaller is now a from-source compile farm.

The decision below is the staff-level judgment: which base for which constraint, and what each one fails at. For a numeric ML scorer the answer is slim-Debian by default, because glibc keeps the prebuilt wheels usable and the size cost over Alpine is modest compared to the build-time and build-risk cost Alpine imposes.

Choosing the base is the first cut. The second cut is removing the build tooling you do install, and there is a non-obvious rule about when you remove it that, gotten wrong, makes the removal accomplish nothing. Docker image layers are strictly additive by design: each instruction commits a filesystem diff as its own read-only layer, and a later layer can shadow a file but cannot reach back and delete bytes from an earlier one. If you apt-get install gcc in one RUN and apt-get remove gcc in a later RUN, the removal records a shadowing entry in the new layer while every byte of gcc still sits in the earlier layer, which still gets transmitted on every push and pull. The image looks smaller in the running container’s view and is exactly as heavy on the wire.

Installing and cleaning in the same RUN layer is the fix, so the layer’s committed diff is install-minus-cleanup and the build-tool bytes were never committed at all. Below, the function treats each RUN as committing a diff: the two-layer version installs in one layer and removes in the next, while the one-layer version does both before the layer is committed. The shipped size is the sum of committed bytes, not what the final filesystem appears to contain.

def shipped_size(layers: list[dict[str, int]]) -> int:
    """Image ships the sum of every committed layer's bytes, additively.
    A later layer cannot subtract bytes already committed by an earlier one."""
    return sum(max(0, sum(diff.values())) for diff in layers)


# Two layers: install gcc (+300 MB), then 'remove' it in a SEPARATE layer.
# The removal layer cannot delete the earlier layer's bytes.
two_layers: list[dict[str, int]] = [
    {"base": 120, "gcc": 300},  # layer 1: base + installed gcc
    {"shadow_gcc": 0},  # layer 2: records a deletion, frees nothing on the wire
]

# One layer: install AND clean before the layer is committed.
# The committed diff never contained gcc.
one_layer: list[dict[str, int]] = [
    {"base": 120, "net_after_cleanup": 0},  # built + cleaned in the same RUN
]

print(f"install + remove-in-later-layer ships: {shipped_size(two_layers)} MB")
print(f"install + clean-in-same-layer ships:  {shipped_size(one_layer)} MB")

The two-layer version still ships the 300 MB of gcc even though the running container cannot see it, because the bytes live in a committed lower layer that every pull copies. Same-layer cleanup is the only version that actually reclaims the space. That additive-layer property is also the reason the next section exists: if you cannot delete a build tool out of an earlier layer, the way to keep it out of the shipped image is to never let it touch the shipped image’s layer chain at all, which is exactly what a multi-stage build does.

def shipped(layers: list[dict[str, int]]) -> int:
    """Sum the committed bytes across layers. Later layers cannot subtract earlier bytes."""
    return 0  # TODO: sum each layer's diff, floored at zero per layer


# gcc is 280 MB; the apt cache it leaves is 90 MB; the runtime base is 120 MB.
# Version A: install (base+gcc+cache) in layer 1, 'remove gcc' in layer 2 (a shadow, frees nothing).
separate: list[dict[str, int]] = [
    {"base": 120, "gcc": 280, "apt_cache": 90},
    {"shadow_gcc": 0},
]
# Version B: install gcc, build the wheel, purge gcc AND the apt cache, all in one RUN.
same_layer: list[dict[str, int]] = [
    {"base": 120},  # TODO: what does the single committed diff actually contain?
]

print("separate:", shipped(separate), "MB | same-layer:", shipped(same_layer), "MB")

def shipped(layers: list[dict[str, int]]) -> int:
    """Sum the committed bytes across layers. Later layers cannot subtract earlier bytes."""
    return sum(max(0, sum(diff.values())) for diff in layers)


separate: list[dict[str, int]] = [
    {"base": 120, "gcc": 280, "apt_cache": 90},
    {"shadow_gcc": 0},
]
same_layer: list[dict[str, int]] = [
    {"base": 120},  # gcc built the wheel and was purged before this layer committed
]

print(f"separate-layer cleanup ships: {shipped(separate)} MB")
print(f"same-layer cleanup ships:     {shipped(same_layer)} MB")
print(
    f"bytes the later 'remove' failed to reclaim: {shipped(separate) - shipped(same_layer)} MB"
)

Multi-stage builds: build fat, ship thin

After same-layer cleanup, the tempting belief is that you have done all you can in one Dockerfile: install lean, clean in place, choose a slim base, and the image is as small as it gets. That belief has a ceiling, and the ceiling is the single-stage structure itself. A single-stage build leaves every build-time layer in the final image because the final image is that one stage’s layer chain, and there is nowhere else for gcc, the -dev headers, and the package caches to live. You can clean within a RUN, but you cannot remove the cost of having had a builder at all when the builder and the runtime are the same chain. Same-layer cleanup minimizes one layer; it does not change the fact that the chain that built the wheel is the chain that ships.

A multi-stage build breaks that constraint by defining more than one FROM. Each FROM begins a fresh, independent stage with its own layer chain, and the stages are not connected by default: they are severed. The builder stage starts from a base that has the compilers, installs the locked dependencies, and builds whatever wheels need building, accumulating fat layers freely because none of them are going to ship. The runtime stage starts from a clean slim base, a brand-new chain with zero builder layers in it, and pulls across only the finished artifacts it names. COPY --from=builder <src> <dst> reaches into the builder’s finished filesystem and copies only the named paths, the installed site-packages and the model artifact, into a single new layer on the runtime chain. It cherry-picks files across an otherwise-severed boundary; it does not merge the stages.

What makes this a real size cut and not bookkeeping is the layer model itself. The shipped image is whatever layers belong to the last stage. The builder stage’s layers are simply never part of the runtime chain, so they are never in the image manifest, never pushed to the registry, never pulled on deploy: they are discarded the moment the build finishes. The compiler, the headers, the build caches, the intermediate object files: all of it stays behind in a stage that exists only on the build machine. This is documented to cut a final image by hundreds of megabytes, and because deployment latency on most systems is gated on how many bytes the registry has to move, cutting the image cuts the deploy time proportionally. The named failure mode this fixes is the one from the opening: an image that carried a full compiler toolchain into production because the dependencies needed it to build and never at run time. Split into a builder and a slim runtime stage that copies only the built packages, the image drops by an order of magnitude with no behaviour change, and the registry push that dominated CI drops to seconds.

Seeing the structure as two chains and one crossing arrow is worth it, because the whole insight is what crosses the boundary versus what is thrown away.

package "builder stage  (discarded, never ships)" {
  [base + gcc + dev headers] as bbase
  [installed site-packages + built wheels] as bpkgs
  [apt cache + object files] as bjunk
  bbase --> bpkgs
  bbase --> bjunk
}

package "runtime stage  (the shipped image)" {
  [slim base (glibc)] as rbase
  [COPY --from=builder: site-packages + model] as rcopy
  [ENTRYPOINT python -m score] as rentry
  rbase --> rcopy
  rcopy --> rentry
}

bpkgs --> rcopy : COPY --from=builder
note bottom of bjunk : compiler, headers, caches\nstay in the builder, not in the manifest

The arrow is the only thing that crosses; everything in the builder box that the arrow does not touch is discarded. That is the difference between a multi-GB single-stage image and a few-hundred-MB runtime image built from the same source.

The severed boundary has a failure mode of its own, and it is this lesson’s first section come back around. COPY --from carries only the paths you name — and a system shared library the builder installed with apt-get does not live in site-packages; it lives in the OS’s library directories, which the copy never mentions. So the wheel that imported cleanly in the builder (where the .so was present) dies in the runtime stage with the exact class-three missing-shared-library error from the start of this lesson — reintroduced by the multi-stage split itself, because the runtime stage is a fresh chain that inherited nothing. The fix is the same one as before, applied on the right side of the boundary: the runtime stage gets its own apt-get install of the runtime shared libraries (not the -dev headers — those were build-time), diagnosed with the same ldd. Two quieter versions of the same boundary mistake: the builder and runtime stages must agree on the Python version, because site-packages lives at a versioned path (python3.12/site-packages) and a copy across mismatched versions lands packages where the runtime interpreter never looks; and console-script entry points installed to the builder’s bin/ do not ride along with a bare site-packages copy. The multi-stage cut is real, but the boundary is a contract you write by hand — everything the runtime needs must be named across it, and the class-three taxonomy from section one is the checklist for what you forgot.

This cut does not stand alone. It composes with the two earlier hardening passes, and the composition is where the order-of-magnitude number comes from. The builder stage still benefits from the lock-first layer ordering from the build-reproducibility lesson, so a code change does not re-resolve the dependency graph. The runtime base is the size floor from the previous section, so the slim glibc base sets how small the shipped chain can be. And the multi-stage discard removes the builder’s chain entirely. The size cut is the arithmetic of all three: lock-first caching keeps the build fast, the slim base sets the floor, and the multi-stage boundary throws away everything above that floor that runtime never uses.

Verifying it means computing the shipped size as only the runtime stage’s layers and confirming the builder’s inventory contributes nothing. Below, each stage carries a layer inventory; the shipped image is the sum of the runtime chain alone, and the builder’s fat layers are present in the build but absent from what ships.

def shipped_image_mb(stages: dict[str, list[int]], last_stage: str) -> int:
    """Only the LAST stage's layers ship. Other stages exist on the build host
    and are discarded -- never in the manifest, never pushed, never pulled."""
    return sum(stages[last_stage])


stages: dict[str, list[int]] = {
    # builder: slim base + gcc/headers + the resolved+built dependency tree + caches
    "builder": [120, 280, 850, 95],
    # runtime: clean slim base + one COPY --from layer (site-packages + model) + entrypoint
    "runtime": [120, 240, 1],
}

total_built: int = sum(sum(layers) for layers in stages.values())
shipped: int = shipped_image_mb(stages, last_stage="runtime")

print(f"bytes built across all stages: {total_built} MB")
print(f"bytes that actually ship:      {shipped} MB")
print(f"discarded with the builder:    {total_built - shipped} MB")

The builder’s base, compilers, and caches are built and then thrown away; what ships is the runtime chain alone. The resolved dependency tree in the builder did not vanish: its installed result crossed the boundary as the COPY --from layer, while the compilers that produced it stayed behind. That is the entire trick: build fat, copy the finished artifacts, ship thin.

def shipped(layers: list[int]) -> int:
    """Sum the layers that actually ship."""
    return 0  # TODO


# Single stage: base + gcc + installed packages + apt cache + model, all in one chain.
single_stage: list[int] = [120, 300, 260, 90, 5]

# Multi-stage builder (does NOT ship): base + gcc + installed packages + cache
builder: list[int] = [120, 300, 260, 90]
# Runtime stage: slim base, then ONE COPY --from layer carrying only packages + model.
# TODO: build the runtime chain -- slim base (120) + copied (packages 260 + model 5) + entrypoint (1)
runtime: list[int] = []  # TODO

print(
    "single-stage:", shipped(single_stage), "MB | multi-stage:", shipped(runtime), "MB"
)

def shipped(layers: list[int]) -> int:
    """Sum the layers that actually ship."""
    return sum(layers)


single_stage: list[int] = [120, 300, 260, 90, 5]

builder: list[int] = [120, 300, 260, 90]  # discarded, never ships
runtime: list[int] = [
    120,
    260 + 5,
    1,
]  # slim base + COPY --from(packages+model) + entrypoint

print(f"single-stage ships:  {shipped(single_stage)} MB")
print(f"multi-stage ships:   {shipped(runtime)} MB")
print(f"left in the builder: {shipped(single_stage) - shipped(runtime)} MB")

Summary

• “Works on my machine” is not carelessness; it is one of five missing-environment failure classes (missing library, wrong version, missing OS shared library, dev-only config, dev-only file), and the class names the fix before you touch the Dockerfile.
• The missing OS shared library is the class a slim base most often exposes, because prebuilt wheels link against system .so files; the fix is apt-get install in the base layer, diagnosed with ldd, never a retreat to the fat base.
• The base image is the size floor: slim-Debian (glibc) is the ML default because the prebuilt manylinux wheels install without compiling; Alpine (musl) does not match those wheels and recompiles numeric packages from source, multiplying build time or failing outright.
• Docker layers are additive: removing a build tool in a later layer leaves its bytes in the earlier one, so install-and-clean must happen in the same RUN to actually reclaim space.
• A multi-stage build defines more than one FROM; the builder stage’s fat layers are never in the runtime chain, so COPY --from=builder cherry-picks only the finished artifacts across and the compilers, headers, and caches are discarded. That is the move that takes a multi-GB image to a few hundred MB with the same model.

Check your understanding:

• Without looking back: name three of the five “works on my machine” failure classes and the specific Docker mechanism that closes each.
• Why is Alpine the wrong default for a numeric ML image even though it produces the smallest base?
• A Dockerfile installs gcc in one RUN and removes it in a later RUN, yet the pushed image is no smaller. What property of layers explains that, and what is the fix?
• In a multi-stage build, what exactly does COPY --from=builder move across, and why do the builder stage’s layers never reach the registry?