Project — Python for ML Engineers

Project Build this yourself, to the spec and rubric below — the module's standard. You build it on your own dataset, alongside the lessons.

Build a clean, importable and runnable Python package for the Adult / Census Income dataset (predict whether a person earns >50K), in your own GitHub repo. It is the skeleton the rest of Phase 1 builds on — you carry this same repo through the M2–M4 projects — so the bar is maintainability, not cleverness.
Dataset Adult / Census Income — different from the lessons’ Lending Club scorer, so finishing it proves the technique transferred, not that you replayed the lesson
Start from Fork dutchengineer-org/phase1-starter — an empty package shell you fill in (do not start from a blank directory); the data comes from ml_pipeline.datasets.load_adult, not a bundled file
Aim for The reference package ml-pipeline-starter — read it to see what good looks like; do not fork it (it is built on Lending Club, yours is the same structure on Census)
Done when It passes every line of the rubric below, then you push to GitHub

Start here

  1. Fork the starter → dutchengineer-org/phase1-starter, clone it, pip install -e .
  2. Work the 5 tasks below in order — each maps to a lesson you just finished
  3. Check yourself against the rubric, then push and submit

The finished package should let another engineer clone your repo, install it once, and both import your code (to reuse a function) and run it (python -m census_pipeline, to score a record) — without anything surprising happening.

The tasks

1. Stand up the package (from Lessons 1–2)

  • The forked starter already ships the standard layout as empty files: pyproject.toml, src/census_pipeline/ with __init__.py, the modules (data.py, features.py, model.py), and __main__.py. Fill them in; do not start a second layout beside it.
  • data.py is your data-loading seam: get the Adult / Census Income frame from the source with from ml_pipeline.datasets import load_adult (the “go get the data” step — a stand-in for the database query a real system would run), and hand it back. You build the module; the package provides the data. __main__.py runs a baseline score end to end.
  • pip install -e . works, and python -m census_pipeline runs your scorer.

2. Make the imports robust (from Lesson 4)

  • No module does real work at import time: importing any module must not load data, train, or write files. The “do the run” code lives behind if __name__ == "__main__":.
  • The package imports cleanly from any directory after the editable install (no sys.path hacks).

3. Make the data contracts solid (from Lesson 5)

  • Add typed data contracts at the boundary: a dataclass or pydantic model for a record, with correct type hints (dict[str, float], str | None, etc.).
  • The serving-style entry point validates input through that contract: a malformed field is rejected with a clear error, not passed silently into the model.

4. Make it reproducible (from Lesson 6)

  • Pin dependencies with a lock file (uv lock) so a fresh clone resolves the same environment.
  • A teammate cloning the repo and following your README gets the same result you do.

5. Configure it through one typed settings object (from Lesson 5, applied to config)

  • Add a settings.py next to your modules with a Settings class built on pydantic-settings (pip install pydantic-settings), the same pydantic that validates your records in Task 3, now pointed at your runtime config. Two fields are enough: the data path as a required field (no default), and the split seed as an optional one:

    from pydantic_settings import BaseSettings

class Settings(BaseSettings):
    census_data_path: str        # required: missing -> refuses to start, names the field
    random_state: int = 42       # optional: the default lives here, once

  • __main__.py constructs Settings() once and passes the values into your functions as arguments. No other module reads os.environ; the library functions stay pure and testable, and the environment is read in exactly one reviewable place.

  • Prove the fail-fast: run python -m census_pipeline with CENSUS_DATA_PATH unset and confirm it exits with a validation error naming the field, instead of running against a fallback.

Hints
  • Start from the src/ layout: it forces you to install the package to import it, which is exactly what makes “works from any directory” true.
  • The fastest way to catch an import side effect: open a Python REPL and import census_pipeline.model. If anything prints, loads, or writes, you have a side effect to move behind the __main__ guard.
  • A mutable default argument (def f(x, acc=[])) is the classic trap: check every default in your signatures.

Rubric — your project is done when

This is the standard the module holds you to (each bar maps to the lesson that taught it):

  • Imports without side effects — import-ing any module runs no job, loads no data, writes no files. (Lessons 1, 4)
  • Runs as a module — python -m census_pipeline scores a record end to end. (Lessons 1, 4)
  • Typed data contracts at the boundary — a dataclass/pydantic model validates records; malformed input is rejected with a clear error. (Lesson 5)
  • Locked dependencies — a lock file makes a fresh clone reproducible. (Lesson 6)
  • Runtime config is one typed object — a Settings(BaseSettings) in settings.py is the only place the environment is read; a missing required value fails at startup naming the field, and library functions take arguments instead of reading os.environ. (Lesson 5, applied to config)
  • No mutable-default or global-state traps — behavior comes from arguments, not hidden module state. (Lesson 4)

Run the self-audit from the Make It Reproducible lesson against your own repo before you submit: for each rubric line, confirm it holds, and if one fails, name which lesson’s failure mode you reintroduced.

Submit

Use the branch workflow from the git lesson, not commits straight to main. Your first commit establishes main as the working package; do the project work on a branch (git checkout -b m1-package-setup), and open a pull request to merge it back once it meets the rubric. main staying green from the start is what keeps M2 onward building on solid ground.

When your repo meets every rubric line, merge your branch to main, push it to GitHub, and submit the repository URL here. (Submission coming soon.)