Get a CI Pipeline Going Green

I set the Python packaging, CI, and git standards for ML, data engineering, and analytics teams at a company where deployments took months. The first pipeline I inherited had a green check on every commit, and nobody trusted it. The check passed because the runner re-used the same long-lived machine for every job, and that machine had a data file, a stray environment variable, and a globally installed package that the repo had never declared. A model that scored fine in CI scored garbage the first time it ran somewhere else, because the somewhere-else did not have any of that ambient state. The green check was a statement about one machine’s accumulated history, not about the code. The work that followed, which is the work this module is about, was making that check mean something, and it starts with making it run somewhere clean, on every change, in an order where a bad commit finds out it is bad as cheaply as possible.

That is the problem this lesson fixes. The pipeline you build here is the happy path: a push triggers a run, the run installs the exact recorded dependencies, runs the tests for the Lending Club default scorer you serialized back in the Packaging module, builds the model artifact, and reports one green mark. It is deliberately the smallest version, because the next four lessons each open with “the green pipeline from Lesson 1 passed, and then this broke anyway.” They add data tests, a model gate, a safe retrain loop, and a staged rollout. None of that attaches to a pipeline that is only green on your laptop. So this lesson owes them a pipeline that is actually green, on a machine that contains only what the repo declares.

This lesson teaches the principles — isolated jobs, a clean runner, ordered stages that fail fast — and shows them in the two CI systems you are most likely to meet, GitLab CI and GitHub Actions, side by side. The principles are not specific to either; the keyword semantics are, so each pipeline below appears in both, in tabs. The point of the contrast is to make the transferable thing visible: once you can see that a stage in GitLab and a job with needs: in GitHub Actions are the same idea wearing different syntax, you can read or write a pipeline on whichever system a team happens to use. The vocabulary maps almost one to one — pipeline file, job, runner, ordering, artifacts — with one difference that is not cosmetic and that this lesson returns to: GitLab runs on every push by default, while GitHub Actions runs nothing until you declare a trigger, and that default decides whether the gate fails safe or fails open.

A pipeline is a compiled graph of isolated jobs, not a script

A .gitlab-ci.yml file reads like a shell script. It has lines that look like commands, in an order, top to bottom, so the natural assumption is that GitLab runs it the way bash would run a script: one process, one working directory, where a variable set near the top is still set further down and a file written in one step is on disk for the next. That model is wrong, and it is wrong in a way that produces a job which passes locally, passes in the stage before it, and then fails in isolation.

Here is the shape of the wrong model breaking. One stage writes a value, the next stage reads it, the way two lines of a script would share a variable:

# The mental model: one process, state flows downward like a script.
trained_at = None

def stage_test() -> None:
    global trained_at
    trained_at = "2026-06-17T10:00:00Z"   # set in the test stage
    print("test: recorded trained_at")

def stage_build() -> None:
    # build assumes the value test left behind
    print(f"build: artifact stamped {trained_at}")   # reads the leftover

stage_test()
stage_build()   # prints the timestamp — looks fine on a laptop

On a laptop this prints the timestamp and looks correct, which is exactly why the model survives until it does not. In GitLab, stage_test and stage_build do not share a process or a trained_at. Each job is handed to a runner that starts a fresh container, clones the repository into an empty directory, runs the job’s script: lines, and is then torn down. Nothing computed in one job survives into the next. The build job runs in a container where trained_at was never set, so it reads None, and a job that stamps an artifact with None is a corrupted artifact that passed CI green.

The fix is to stop treating the pipeline as a script and start treating it as what it is: a declarative graph that GitLab compiles from the YAML. On a push, the GitLab server parses and validates .gitlab-ci.yml, produces a set of jobs grouped into ordered stages, hands each job to a runner (here a fresh Docker container pulled from the job’s image:), and reports each container’s exit code, where zero is success and non-zero fails the job. The single green or red mark on the commit is the AND of every job’s exit code.

The vocabulary the rest of the module depends on, each grounded once here:

This is the skeleton the whole module fills in. The reason to show it before writing a line of config is that every later gate is a new job in this graph, not a new line in a script, so the boundary between jobs is the thing to understand first.

Here is that graph written out as the actual pipeline file, in both systems. Read each as three independent jobs that install from scratch, not as a script where the install carries forward. The two tabs encode the same graph — lint, then test, then build — so reading them against each other is the fastest way to see which parts are the idea and which parts are the syntax.

# .gitlab-ci.yml — runs on every push BY DEFAULT (the TRIGGER is implicit)
image: python:3.12-slim     # the RUNNER — a fresh, clean container image
stages: [lint, test, build] # ordered STAGES, run top to bottom
lint:
  stage: lint               # cheapest check runs first
  script:
    - uv sync --locked       # install the recorded deps; fail if the lock is stale
    - ruff check .
test:
  stage: test
  script:
    - uv sync --locked
    - pytest                 # run the assertions
build:
  stage: build
  script:
    - python -m build        # produce the artifact (the model package)
# result: GREEN if every stage passed, RED if any failed

# .github/workflows/ci.yml — runs NOTHING until you declare the trigger
on: push                    # the TRIGGER is explicit; omit it and the gate never fires
jobs:
  lint:
    runs-on: ubuntu-latest  # the RUNNER — a fresh, clean VM
    steps:
      - uses: actions/checkout@v4
      - run: uv sync --locked   # install the recorded deps; fail if the lock is stale
      - run: ruff check .
  test:
    runs-on: ubuntu-latest
    needs: lint             # ORDERING is a dependency edge, not a stage list
    steps:
      - uses: actions/checkout@v4
      - run: uv sync --locked
      - run: pytest
  build:
    runs-on: ubuntu-latest
    needs: test
    steps:
      - uses: actions/checkout@v4
      - run: python -m build  # produce the artifact (the model package)
# result: GREEN if every job succeeded, RED if any failed

Reading the two against each other, the mapping is almost word for word: GitLab’s ordered stages: list is GitHub Actions’ needs: edges between jobs (a dependency graph rather than a linear list, which is strictly more expressive but means the same thing for a straight lint→test→build chain); GitLab’s image: is GitHub Actions’ runs-on: plus an explicit actions/checkout step, because GitHub Actions does not put your code in the runner until you ask it to. The one line that is not a rename is the trigger: GitLab’s run-on-push is implicit, and GitHub Actions’ on: push is mandatory — delete it and the workflow is valid YAML that never runs, the failure mode this lesson’s last section is about.

Both blocks above are illustrations, not something this page executes — a CI pipeline only runs on a real CI server with a real runner, not inside a lesson. Where you actually run one is your project: the module project asks you to commit a .github/workflows/ci.yml to your own GitHub repository, and GitHub runs it for real the moment you push, turning the commit green or red. So read these two as the map; your project repo is the territory where the pipeline runs, fails, and teaches you what the green mark is worth.

The detail that bites first is that each of lint, test, and build is its own fresh container. They do not share a process, a filesystem, or an exported variable. uv sync --locked appears in both lint and test not out of carelessness but because the install that ran in lint does not exist in test’s container: the test job starts from an empty checkout and must install again. A build job that expected pytest’s coverage file to be on disk would find nothing there. Anything a later job needs from an earlier one has to cross the job boundary as an explicitly declared artifact — GitLab spells this artifacts:, GitHub Actions splits it into an actions/upload-artifact step in the producing job and actions/download-artifact in the consumer — but both encode the same rule: a job saves its output and a later job fetches it, and nothing crosses the boundary that was not declared. The named failure mode here is the leftover-state job: a job that assumes another job’s filesystem or variables and breaks the moment it runs alone. It is invisible on a laptop because a laptop is one process where state does flow downward.

The non-obvious cost of this isolation is that you pay for the install twice (or three times), and the instinct of a mid-level engineer is to “fix” that by collapsing everything into one job to share the install. That trades the redundant install for a worse property: a single job cannot fail-fast between checks, cannot run its parts in parallel, and gives you one coarse pass/fail instead of three precise ones. The isolation you are paying for is what lets the next four lessons each attach as their own gate. The redundant install is cheap to recover with caching; the lost gate boundary is not.

To make the isolation visible as a failure rather than a footnote, here is a small simulation: each stage is a function given its own fresh namespace, so a value one stage sets is deliberately not visible to the next.

from typing import Callable


def run_job(
    name: str, script: Callable[[dict[str, str]], int], shared_repo: dict[str, str]
) -> int:
    """Simulate one CI job: a FRESH local namespace, only the repo checkout
    is shared. Returns the job's exit code (0 = pass)."""
    job_env: dict[str, str] = {}  # fresh container env -- empty every job
    exit_code = script(job_env)  # the job's own variables die with it
    print(f"  {name}: exit {exit_code} (job_env had {list(job_env.keys())})")
    return exit_code


def lint_job(env: dict[str, str]) -> int:
    env["LINT_RAN"] = "yes"  # set a variable in THIS job only
    return 0


def build_job(env: dict[str, str]) -> int:
    # build tries to read what lint set -- but env is fresh, so it is absent
    if "LINT_RAN" not in env:
        print("    build: LINT_RAN not visible -- jobs do not share env")
        return 1  # non-zero: the leftover-state failure
    return 0


def main() -> None:
    repo = {"src": "score.py"}  # the only thing that crosses jobs
    codes = [run_job("lint", lint_job, repo), run_job("build", build_job, repo)]
    pipeline_pass = all(code == 0 for code in codes)  # AND of every exit code
    print(f"pipeline: {'GREEN' if pipeline_pass else 'RED'}  (codes={codes})")


if __name__ == "__main__":
    main()

The build job exits non-zero because LINT_RAN was never in its environment: the variable lint set lived and died inside lint’s container. The pipeline mark is the AND of the exit codes, so one non-zero job turns the whole commit red. Why does the simulation give build a brand-new empty job_env instead of reusing lint’s? Because that empty dict is the entire point. It is the fresh container, and reproducing it in Python is the only way to see the failure the laptop hides.

The static anatomy of this graph (trigger, stages, jobs as separate boxes, and the single artifact edge that crosses between them) is worth holding as a map before the run unfolds over time.

[push] as trigger
package "stage: lint" {
  [lint job\nimage: python:3.12-slim\nscript: uv sync --locked; ruff] as lint
}
package "stage: test" {
  [test job\nimage: python:3.12-slim\nscript: uv sync --locked; pytest] as test
}
package "stage: build" {
  [build job\nimage: python:3.12-slim\nscript: python -m build] as build
}
trigger --> lint
lint --> test : stage passed
test --> build : stage passed
build ..> test : artifact (the ONLY thing that crosses jobs)

Each job is its own box with its own image: and script:, and the only line that crosses between boxes is the dashed artifact edge; every other dependency is stage ordering, not shared state. That boundary is the containment shape the rest of the module attaches to.

def simulate(stages: list[str]) -> str:
    installed_version: str | None = None
    log: list[str] = []
    for stage in stages:
        # each stage runs in a FRESH job -- nothing from a prior job is here
        if stage == "install":
            local_version = "2.1.0"  # computed inside the install job
            _ = local_version
            log.append("install: resolved 2.1.0")
        elif stage == "build":
            # build is a different job: does it see install's local_version?
            log.append("build: stamping " + str(installed_version))
        else:
            log.append(stage + ": ran")
    return " | ".join(log)


def main() -> None:
    print(simulate(["install", "test", "build"]))  # predict the build line first


if __name__ == "__main__":
    main()

def simulate(stages: list[str]) -> str:
    installed_version: str | None = None  # build's view: never written here
    log: list[str] = []
    for stage in stages:
        if stage == "install":
            local_version = "2.1.0"  # lives only in the install job
            _ = local_version  # used here, gone after
            log.append("install: resolved 2.1.0")
        elif stage == "build":
            log.append("build: stamping " + str(installed_version))
        else:
            log.append(stage + ": ran")
    return " | ".join(log)


def main() -> None:
    print(simulate(["install", "test", "build"]))
    # install -> test -> build; build stamps None, because the version
    # computed in the install job never crossed the job boundary as an artifact.


if __name__ == "__main__":
    main()

Passing on a clean runner is what makes it CI and not a script

The pipeline runs, the jobs are isolated, and the tests pass. The next assumption a competent engineer makes is that a green test result is evidence the code is correct: the suite ran, every assertion held, so the code works. That assumption is true only if the suite ran somewhere that contains nothing the repository did not declare, and a developer laptop is the opposite of that.

Here is the assumption breaking. A test passes because a value happens to be in the environment, and the environment is the laptop’s accumulated history:

import os

def score_threshold() -> float:
    # reads an env var the laptop happens to have, the repo never declared
    return float(os.environ["DEFAULT_CUTOFF"])   # exported months ago in .zshrc

def test_threshold_in_range() -> None:
    assert 0.0 <= score_threshold() <= 1.0       # passes ONLY if the var is set

On the laptop where DEFAULT_CUTOFF was exported into .zshrc months ago, this test passes and the green check appears earned. On a fresh runner that variable does not exist, so score_threshold() raises KeyError and the test errors out: the same code, the same test, a different machine, an opposite result. The green was a property of the laptop, not the code. This is the ambient-state pass: a test that reads undeclared machine state and reports the machine’s history as if it were the code’s correctness. It is the exact failure that made the first pipeline I inherited untrustworthy, with a globally installed package and a stray variable doing the work the repo was supposed to do.

The fix is two halves, and together they are the definition of CI: run on a machine that contains only the repo’s declarations, and run on every change so no one can skip the gate. Neither half is optional. A clean run that only happens when someone remembers is not continuous. A continuous run on a dirty machine is not evidence. The first pipeline failed the clean half; plenty of pipelines fail the continuous half by being a manual button someone presses before a release.

The clean half is handled by construction. The runner starts each job from the declared image: (a fresh container with only what that image ships), clones the repo into an empty directory, and installs exactly the recorded dependency graph. The word “exactly” is load-bearing and is why this track locked dependencies in the Packaging module: installing from the lock file, not from loosely-pinned ranges, is what makes “the same versions every run” true.

There is a precise trap in how you install from the lock, and getting it wrong silently re-introduces the ambient-state problem inside CI. The relevant tunable is the staleness check, and uv splits it across two flags:

The signal for which to reach for: in CI, use --locked so a stale lock fails the job loudly; reach for --frozen only when you have a deliberate reason not to check (offline, or a job that intentionally pins to an older lock). The concept-level mistake is assuming --frozen is the safe default because “frozen sounds strict”; it is the opposite of the staleness guard. The strict one is --locked.

The trigger half is load-bearing in the opposite direction, and this is the one place the two systems genuinely disagree rather than merely rename. GitLab runs .gitlab-ci.yml on a push, a merge request, or a schedule by default, and workflow: / rules: only ever narrow that default. The gate is on unless someone writes an explicit, reviewable rule to turn it off. That asymmetry is a safety property: an on-by-default gate that you turn off by exception fails safe, because turning it off is a visible diff someone reviews. GitHub Actions defaults the other way. A workflow runs nothing until its on: key declares a trigger, so a .github/workflows/ci.yml that is missing or mistypes on: push is valid YAML that simply never fires — the gate is silently off, and nothing in a pull request shows that it is, because the absence of a run looks identical to a run that has not started yet. That is the fail-open case: the first time someone forgets the trigger, every check the pipeline was supposed to enforce is skipped with no red mark. The lesson is not that one system is safer than the other; it is that the direction of the default is a fact you have to know per system, because it decides whether forgetting a line leaves the gate on or off. On GitHub Actions, treat on: as the most important line in the file and confirm a run actually appears on the commit.

To show the green as visibly a statement about the environment, here is the same test run twice: once with the variable present, once on a clean runner without it.

from typing import Callable


def run_on_runner(env: dict[str, str], test: Callable[[dict[str, str]], None]) -> str:
    """A runner is the environment its container ships with."""
    try:
        test(env)
        return "GREEN"
    except KeyError as missing:
        return f"RED (missing {missing} -- undeclared ambient state)"


def threshold_test(env: dict[str, str]) -> None:
    cutoff = float(env["DEFAULT_CUTOFF"])  # reads the environment
    assert 0.0 <= cutoff <= 1.0


def main() -> None:
    laptop = {"DEFAULT_CUTOFF": "0.5", "PATH": "/usr/local/bin"}  # accumulated state
    clean_runner: dict[str, str] = {"PATH": "/usr/bin"}  # only what's declared

    print("laptop:      ", run_on_runner(laptop, threshold_test))
    print("clean runner:", run_on_runner(clean_runner, threshold_test))


if __name__ == "__main__":
    main()

The laptop prints GREEN and the clean runner prints RED for the identical test, which makes the point sharper than any assertion: the difference is the machine, not the code. Why does the clean runner catch the bug the laptop hides? Because the clean runner contains only what the repo declared, so a dependency on undeclared state has nowhere to hide: the KeyError is the bug surfacing the instant the ambient state is removed. The fix is to make the cutoff a declared input (a config file in the repo, installed deps from the lock), not a variable the runner happens to carry. And when a value legitimately is per-environment runtime config rather than a repo constant, it still never gets read as a bare os.environ[...] inside a function: it goes through the typed Settings object from the packaging module, where it either carries a declared default or fails at startup with the field named — so on a clean runner the misconfiguration is a named boot error, not a KeyError three frames into a test.

There is one more reason the runner, not the developer, must run the suite. As soon as the test suite takes more than a few seconds, developers stop running it locally. The friction is enough that “I will run it before I push” quietly becomes “I pushed.” This is why CI exists as a server-side gate rather than a pre-push habit: the habit decays exactly when the suite gets valuable enough to be slow.

def is_trustworthy(passed: bool, ran_on_clean_runner: bool, triggered_by: str) -> bool:
    # starter: trusts any passing run -- this is the bug
    if passed:
        return True
    return False


def main() -> None:
    print(
        is_trustworthy(True, ran_on_clean_runner=False, triggered_by="manual")
    )  # should be False
    print(
        is_trustworthy(True, ran_on_clean_runner=True, triggered_by="push")
    )  # should be True


if __name__ == "__main__":
    main()

def is_trustworthy(passed: bool, ran_on_clean_runner: bool, triggered_by: str) -> bool:
    # both halves of CI must hold: clean environment AND automatic trigger
    automatic = triggered_by in ("push", "merge_request", "schedule")
    return passed and ran_on_clean_runner and automatic


def main() -> None:
    print(
        is_trustworthy(True, ran_on_clean_runner=False, triggered_by="manual")
    )  # False
    print(is_trustworthy(True, ran_on_clean_runner=True, triggered_by="push"))  # True
    print(
        is_trustworthy(True, ran_on_clean_runner=True, triggered_by="manual")
    )  # False


if __name__ == "__main__":
    main()

Order by cheapest-failure-first

The stages are isolated and the runner is clean, so the remaining decision looks cosmetic: which stage goes first. The plausible instinct is to order stages by dependency or by what feels logical, building the thing first and then testing it, or to treat the order as tidiness that does not affect correctness. Both miss what the order actually controls. The order is the lever that sets how fast a bad change finds out it is bad, and therefore how much compute a bad change burns before it is rejected.

A note on stages before going further. The build stage in this lesson’s .gitlab-ci.yml packages the model artifact (python -m build), and packaging is cheap. The expensive, side-effecting work in an ML pipeline is the model fit itself, which the next lessons add as a train stage between test and build: train consumes data and produces a model, and build packages that model for distribution. The ordering argument below is about that expensive train stage, because it is the one whose placement controls real compute cost. Where this section says train, it means the side-effecting fit stage, distinct from the cheap build packaging stage you wrote in §1.

Here is the wrong order’s cost made concrete. Three stages, the expensive fit first, and a one-line syntax error in the commit:

# Wrong order: train the model first, lint last.
def train() -> int:
    expensive_fit()          # 4 minutes of compute, writes an artifact
    return 0

def lint() -> int:
    raise SyntaxError("missing colon on line 12")   # a 2-second check would catch this

# stages run in declared order; train runs BEFORE lint catches the typo
train()   # pays 4 minutes...
lint()    # ...only to fail on a typo a 2-second check would have caught first

The syntax error is the cheapest possible failure to detect: a linter finds it in about two seconds without running anything. Put the train stage first and every commit with a typo pays the full multi-minute fit before the lint that would have caught it even runs. Stages run in strict declared order, and a stage starts only if every job in the prior stage exited zero. That is fail-fast, enforced by GitLab’s scheduler, not by convention. So the order directly sets the expected cost: each stage has a cost (wall-clock plus compute) and a failure probability, and total expected cost is minimized by running the high-probability-cheap checks before the low-probability-expensive ones. Lint and unit tests fail often and finish in seconds; a model fit fails rarely and runs for minutes. Cheapest-failure-first is the ordering that puts the frequent, cheap rejections before the rare, expensive ones.

The response curve on stage ordering, the same three stages with different placement of the expensive one:

The signal for which to reach for: default to cheapest-failure-first; introduce needs: only when two checks are genuinely independent and you have measured the prior stage as the bottleneck. The trap with needs: is that it looks like free parallelism, but it converts an ordering guarantee into a hand-maintained dependency graph: needs: jobs start immediately after their own dependencies finish, even if other jobs in earlier stages are still running, which means a forgotten edge silently lets an expensive job start early. You traded a guarantee the scheduler enforced for one you now maintain by hand.

There is a second, sharper reason order matters for ML specifically, and it is the one that separates this from ordering a web app’s CI. Most stages are pure: lint, type-check, and unit tests read code and produce a pass/fail, with no lasting effect. The train stage is not pure: it consumes data and compute and produces an artifact with side effects. Running it on a commit a cheap check would have rejected does not only waste minutes; it can write a known-bad artifact into a cache or registry that later runs read as if it were good. This is the poisoned-artifact failure: a side-effecting stage that ran on input the pipeline should have rejected, leaving downstream state corrupted. Cheapest-failure-first guarantees the side-effecting stage only ever runs on input that already cleared everything cheaper, which is why the principle is not merely an optimization for ML. It is a containment boundary around the one stage that can leave a mess.

This ordering principle is also where the rest of the module attaches. There are three test layers, and each is a stage that obeys the same rule. Code tests are this lesson, the test stage. Data tests are the next lesson: a new stage after the code tests and before the model fit, because validating the data is cheaper than training on bad data and discovering the problem in the model. Model tests are the lesson after that: a gate before promotion, because checking a model is cheaper than shipping a bad one. Each later gate is the cheapest check that catches its class of failure, inserted before the expensive thing it protects.

The run unfolding stage by stage, and the cost of where the failure lands, is the thing to watch here, because a sequential short-circuit is hard to see in prose.

The simulation below makes the cost visible as a number. A four-stage pipeline runs cheapest-first, an early stage fails, and the log shows the expensive stage never ran; then the same bug runs through the reversed order and pays the full cost.

from typing import Callable

# (name, cost_in_seconds, returns_exit_code)
Stage = tuple[str, int, Callable[[], int]]


def fails() -> int:
    return 1  # this stage finds the bug


def passes() -> int:
    return 0


def run_pipeline(stages: list[Stage]) -> tuple[bool, int]:
    """Fail-fast: stop at the first non-zero exit, summing only the cost paid."""
    cost_paid = 0
    for name, cost, job in stages:
        cost_paid += cost
        code = job()
        print(f"  {name}: ran ({cost}s), exit {code}")
        if code != 0:
            print(f"  -> fail-fast: stages after {name} never ran")
            return False, cost_paid
    return True, cost_paid


def main() -> None:
    # The bug is a lint failure. Cheapest-first catches it for 2 seconds.
    cheap_first: list[Stage] = [
        ("lint", 2, fails),
        ("test", 8, passes),
        ("train", 240, passes),
    ]
    # Reversed: train runs first and pays 240s before lint catches the same bug.
    expensive_first: list[Stage] = [
        ("train", 240, passes),
        ("test", 8, passes),
        ("lint", 2, fails),
    ]

    print("cheapest-failure-first:")
    _, cost1 = run_pipeline(cheap_first)
    print(f"  cost paid: {cost1}s\n")
    print("expensive-first (same bug):")
    _, cost2 = run_pipeline(expensive_first)
    print(f"  cost paid: {cost2}s")
    print(f"\nsame bug, {cost2 // cost1}x the wasted compute from order alone")


if __name__ == "__main__":
    main()

The cheapest-first run pays two seconds to reject the bug; the reversed run pays two hundred and fifty seconds for the identical failure, because the lint check sits behind the full fit. The ratio is the cost of getting the order wrong, and in the reversed case the train stage also ran, on a commit that should never have reached it. Why does the reversed pipeline still pass its train stage before failing? Because the bug is a lint bug, and with lint last the expensive train stage completes and writes its artifact before the cheap check ever runs, which is exactly the poisoned-artifact risk the ordering prevents.

def order_stages(stages: list[dict]) -> list[str]:
    # each stage: {"name": str, "seconds": int, "fails_often": bool}
    # starter: returns them in the given (scrambled) order -- fix this
    return [s["name"] for s in stages]


def main() -> None:
    scrambled = [
        {"name": "train", "seconds": 240, "fails_often": False},
        {"name": "type-check", "seconds": 3, "fails_often": True},
        {"name": "unit-test", "seconds": 8, "fails_often": True},
        {"name": "build-artifact", "seconds": 30, "fails_often": False},
    ]
    print(order_stages(scrambled))


if __name__ == "__main__":
    main()

def order_stages(stages: list[dict]) -> list[str]:
    # sort key: failing-often first (so not fails_often -> True sorts later),
    # then by cost ascending so the cheapest of an equal class runs first.
    ordered = sorted(stages, key=lambda s: (not s["fails_often"], s["seconds"]))
    return [s["name"] for s in ordered]


def main() -> None:
    scrambled = [
        {"name": "train", "seconds": 240, "fails_often": False},
        {"name": "type-check", "seconds": 3, "fails_often": True},
        {"name": "unit-test", "seconds": 8, "fails_often": True},
        {"name": "build-artifact", "seconds": 30, "fails_often": False},
    ]
    print(order_stages(scrambled))
    # ['type-check', 'unit-test', 'build-artifact', 'train']


if __name__ == "__main__":
    main()

Summary

• A CI pipeline is a declarative graph the server compiles from a YAML file (.gitlab-ci.yml in GitLab, .github/workflows/ci.yml in GitHub Actions), not a script run top to bottom. Each job runs in its own fresh container or VM with an empty checkout, so filesystem changes and variables set in one job do not survive into the next; anything a later job needs must cross the boundary as a declared artifact (artifacts: in GitLab, upload/download-artifact in GitHub Actions). The vocabulary maps almost one to one between the two; the real difference is the trigger default — GitLab runs on push by default, GitHub Actions runs nothing until on: is declared — which decides whether a forgotten line leaves the gate on or off.
• A green check is evidence about the code only when it runs on a clean runner that contains only the repo’s declarations and runs automatically on every change. A test that reads undeclared ambient state reports the machine’s history, not the code’s correctness.
• Install from the lock with uv sync --locked so a stale lock fails the job loudly; --frozen skips the staleness check and can silently install a drifted graph.
• Stages run in strict declared order and fail-fast stops the pipeline at the first non-zero exit, so order is the lever that sets expected cost: cheapest-failure-first puts frequent, cheap checks before rare, expensive ones, and keeps the side-effecting train stage from ever running on input a cheaper check would reject.
• The module’s later gates attach as new stages on this graph: code tests (this lesson), then data tests, then the model gate, each the cheapest check that catches its class of failure, inserted before the expensive thing it protects.

Check your understanding:

• Without looking back: why does a variable set in the test job not exist in the build job, and what is the one mechanism that carries a value between them?
• A test passes on your laptop and fails on the CI runner with a KeyError. What does that tell you about the test, and why is the runner’s result the trustworthy one?
• Why run lint before model training when both can fail the build, and what does fail-fast stop from running? What second risk, specific to a train stage, does putting it last contain?