Version Control Your Package with Git

You find yourself staring at a module you refactored an hour ago. It worked before you touched it. Now it does not, you have saved over the good version half a dozen times since, and the only record of the state that worked is your memory of it. That is not enough. You retype what you think it used to be, you guess, and you hope.

I have been there, and it is the afternoon that turned git from a thing I knew about into a habit. That is the gap this lesson closes. The package you built in the previous two lessons imports cleanly and runs from anywhere, but it has no past. Every edit overwrites the last, it lives in exactly one place, and no one else can see it. Git gives your code a history you can walk back into and a copy other people can reach.

So here is the plan. First we will put that package under git and push it to GitHub, then walk the everyday loop you will repeat for the rest of the course. Then we will get into branches, which is the part that separates “I can save my code” from “I can work on it safely.”

What git is, and the everyday loop

Git is version control. It records the history of your files as a series of commits you can return to, and it syncs that history to a remote, GitHub, so the work is backed up and shareable. The everyday loop is four steps: edit, add, commit, push. You’ll run it hundreds of times. It’s worth understanding rather than memorizing, because each step moves your work between a different place git keeps it, and almost every confusing git moment is really a question of which place your change is sitting in.

There are three of those places. Your edits live in the working tree, the actual files on disk. git add copies a snapshot of the changes you choose into the staging area (also called the index), which is why you can commit some edits and leave others for later. Staging is the act of choosing what the next commit will contain. git commit then takes exactly what is staged and writes it as a permanent commit object: a snapshot, a message, and a pointer back to its parent commit, all identified by a content hash. Because every commit points at its parent, your history is a chain you can walk backward, and that chain is both your undo button and your audit trail.

All of that is still on your machine. git push is the separate step that sends your local commits to the remote so they’re backed up and visible to others. That separation is the source of the single most common git confusion, and once you see it, the confusion evaporates. “I committed but my teammate cannot see it” means committed-but-not-pushed. “I committed but my new file is not in it” means edited-but-not-staged. The boundaries between working tree, index, local history, and remote are exactly the things you cannot see when you stare at your files, so naming them gets you most of the way there.

There is a subtler version of that second confusion worth heading off, because it catches people who did stage. git add snapshots the file’s content at the moment you run it. It is not a live link to the file. So if you git add a file, then keep editing it, then commit, the commit contains the version from the add, not the newer one on disk. The file shows as both staged and modified, and the fix you just typed is sitting unstaged. The symptom is “I staged it and committed and the change still is not in the commit,” which looks impossible until you know the index holds a snapshot, not a pointer. The habit that avoids it: run git status before committing, and treat a file listed under both “staged” and “not staged” as a signal to add again.

The same four places also tell you how to undo, which is the question you’ll actually be asking under pressure, and the answer depends entirely on which boundary the mistake is sitting at. Staged the wrong file but haven’t committed? It’s in the index. git restore --staged <file> unstages it without touching your edits. Made an edit you want to throw away entirely? That’s in the working tree. git restore <file> discards it, and this one does lose work, so it’s the one to be sure about. Committed something you didn’t mean to, but haven’t pushed? It’s in local history only. git reset --soft HEAD~1 rewinds the commit while keeping the changes staged, so you can recommit them correctly. Already pushed? Now the bad commit is on the remote where others may have it, so you don’t rewrite history out from under them. You add a new commit that reverses it with git revert. Here’s the pattern that matters: there’s no single “undo” in git, there’s an undo per boundary, and naming where the mistake lives tells you which one is safe. The line you never cross is rewriting history that other people have already pulled.

Here’s the loop run for real on the Lesson-1 package. The local steps (init, add, commit) actually execute, and the git log confirms the commit landed. The GitHub steps (remote add, push) are shown as the exact commands, since putting it on GitHub needs an account this sandbox does not have:

#!/usr/bin/env bash
# Lesson 2.1 — turn the Lesson-1 package into a git repo and put it on GitHub.
#
# The local git workflow (init, add, commit) RUNS here and is verified. The
# GitHub-touching steps (remote add, push) cannot run in CI — no network or auth —
# so they are echoed as the exact commands to run, not executed. Everything printed
# is real; only the remote lines are non-executing.
set -euo pipefail

# Work in a throwaway dir so the example never touches the real repo.
work="$(mktemp -d)"
trap 'rm -rf "$work"' EXIT
cd "$work"

# A minimal stand-in for the Lesson-1 package.
mkdir loanpkg
cat > loanpkg/__init__.py <<'PY'
PY
cat > loanpkg/data.py <<'PY'
def load_loans():
    return "loans"
PY

# --- Local workflow: runs and is verified -----------------------------------
git init -q
git add .
git commit -q -m "Initial commit: loanpkg"
echo "commit created:"
git log --oneline

# --- Remote workflow: echoed, not executed (no GitHub in CI) -----------------
echo
echo "# Next, on GitHub (not run here — needs auth + network):"
echo "git remote add origin git@github.com:<you>/loanpkg.git"
echo "git push -u origin main"

git init turns the directory into a repository, git add . stages the whole package, and git commit writes the first commit object, which the log then shows by its hash. Everything up to push is local and reversible. push is the line that makes it real for anyone but you.

That git add . deserves a hard warning, because in an ML repository it’s how the worst git mistakes happen. It stages everything in the directory, and an ML directory is full of things that must never be committed. Three categories, in order of how much pain they cause. Secrets are the unrecoverable one: a .env with an API key, a credentials file, a token in a config. Once a secret is committed and pushed, it’s in the history forever. Deleting it in a later commit does not remove it from the past, so the only real fix is to rotate the key, and the cost of forgetting is a leaked credential in a public repo. Large and regenerable artifacts are the bloat one: your .venv/ (hundreds of megabytes you can recreate with one install), __pycache__/, and especially the data and model files this curriculum produces. A 2 GB training CSV or a serialized model committed once stays in the repo’s history forever, so every clone forever downloads it even after you delete it. Git is built for source code, small text files it can diff line by line, and it’s the wrong tool for enormous binary datasets, which is its own discipline (data versioning) you meet later. Local noise is the annoyance one: .DS_Store, editor configs, notebook checkpoints. The fix for all three is a .gitignore file listing patterns git refuses to stage (.venv/, __pycache__/, *.csv, *.joblib, .env), committed at the start so git add . is safe to run. We draw the full commit-vs-ignore line deliberately in the reproducibility lesson. The habit to build now is to write the .gitignore before your first git add ., never after, because the cleanest moment to keep a file out of history is before it ever enters it.

Without scrolling up: what is the difference between git add and git commit, and between git commit and git push? Which boundary is “my teammate cannot see my change” sitting on?

symptoms = {
    "my new file is not in the commit": "???",
    "teammate cannot see my commit": "???",
    "I want yesterday's working state back": "???",
}
# Fill each value: which boundary, and the command. (staged? pushed? history?)
print(symptoms)

symptoms = {
    "my new file is not in the commit": "not staged -> git add <file>",
    "teammate cannot see my commit": "committed but not pushed -> git push",
    "I want yesterday's working state back": "walk history -> git checkout <commit>",
}
for symptom, fix in symptoms.items():
    print(f"{symptom}: {fix}")

Branches and a branching strategy

Committing straight to main means every change you make, finished or not, working or broken, lands on the one branch everything else depends on. A branch is how you avoid that: an isolated line of work where you build a change, open a pull request, and merge it back to main only once it’s ready. This is the workflow the module projects expect, and the one the CI module is built directly on top of.

What a branch actually is turns out to be smaller than it sounds. A branch is a movable pointer to a commit, nothing more. main is one such pointer. git checkout -b feature creates a second pointer starting at the same commit. When you commit on feature, only the feature pointer moves forward. main stays exactly where it was. That single fact is the entire reason an unfinished or broken change on a branch cannot destabilize main: main literally has not moved. And because the two pointers still share all the history up to the branch point, git can compute the diff between them, which is precisely what a code review reviews.

A pull request is GitHub’s wrapper around that diff. It shows the change, runs any checks (this is the hook the CI module attaches to), and on approval performs the merge, moving main forward to include the branch’s commits. Which kind of merge you get follows a precise rule, and it’s worth stating exactly because it predicts the failure below. A merge is a fast-forward if and only if main’s current commit is still an ancestor of the branch tip, meaning main has not moved since you branched, so git slides the pointer forward with no new commit. The instant main has moved, the two histories have diverged, main is no longer an ancestor, and git must write a new merge commit with two parents to tie them together. The whole strategy compresses to one line: main always works; new work happens on a branch; the branch merges via a reviewed PR.

The loop so far is one-directional (your work goes out, edit to push), but a shared main moves under you too, and the half that brings other people’s commits in is where conflicts come from. Before you push a branch, or before you start new work, you git pull (fetch the remote’s commits and merge them into yours), because if main has advanced since you branched, you want that divergence handled now, locally, on your terms, rather than discovered as a rejected push later. Most pulls are uneventful: git merges the changes automatically because they touch different lines of different files. A merge conflict is the case it cannot decide. You and a teammate changed the same lines of the same file, and git refuses to guess which version wins. It doesn’t lose anything or break. It pauses the merge, writes both versions into the file between <<<<<<<, =======, and >>>>>>> markers, and hands it to you to resolve by hand. Resolving is exactly that: open the file, delete the markers, keep the lines that should survive (often a blend of both), then git add the file and commit to complete the merge. It’s mechanical once you’ve seen it, and panicking and force-pushing over it is how people lose work that git had safely preserved.

This is also the moment the notebook habits from the last lesson come back to bite. A Jupyter notebook is a JSON file holding code, output, and execution counts, so two people running the same notebook produce wildly different JSON even when the code barely changed, and git, diffing that JSON line by line, reports a conflict it has no hope of merging sensibly. A notebook merge conflict is often unresolvable by hand without corrupting the file. This is one more reason the real pipeline lives in .py modules, which diff and merge cleanly, and the notebook stays as exploration. Text merges. JSON-with-embedded-output does not.

That ancestor rule also names the cost the standard advice hides, because a branch is not free the longer it lives. A feature branch that diverges from main for two weeks accumulates a merge that has to reconcile every change main took in the meantime, and that reconciliation is where a bug someone already fixed on main gets silently re-introduced. Your branch still carries the old line, the merge keeps your side of it, and the fix is undone without anyone editing it back. The defense is not “never branch” but “merge often.” A short-lived branch stays at or near fast-forwardable, so the divergence never grows large enough to hide a regression. The related footgun: never rebase a branch you have already pushed and shared. Rebasing rewrites the commit hashes, so a teammate who already pulled the old hashes now holds a history that disagrees with yours, and the next pull surfaces as a tangle of duplicated commits. Skipping branches is harmless for a solo first commit and becomes a real liability the instant the work is shared or a CI gate sits on the merge.

Here’s a feature branch built, committed, and merged. Watch main sit still the entire time the work is happening, and move only at the merge:

#!/usr/bin/env bash
# Lesson 2.2 — a feature branch, a change, and a merge back to main.
#
# The local branching workflow (branch, commit, merge) RUNS here and is verified:
# main stays untouched until the merge. The push + open-a-PR steps cannot run in CI,
# so they are echoed as the commands to run, not executed.
set -euo pipefail

work="$(mktemp -d)"
trap 'rm -rf "$work"' EXIT
cd "$work"

git init -q
mkdir loanpkg && : > loanpkg/__init__.py
echo "def load_loans(): return 'loans'" > loanpkg/data.py
git add . && git commit -q -m "Initial commit"

main_before="$(git rev-parse main)"

# --- Feature branch: runs and is verified -----------------------------------
git switch -q -c add-scorer
echo "def score(df): return 0.8" > loanpkg/score.py
git add . && git commit -q -m "Add scorer"

# main is untouched while work happens on the branch.
echo "main before merge: $main_before"
echo "main still at:     $(git rev-parse main)  (unchanged — work is on the branch)"

# --- Push + PR: echoed, not executed ----------------------------------------
echo
echo "# Push the branch and open a PR (not run here — needs GitHub):"
echo "git push -u origin add-scorer"
echo "gh pr create --base main --head add-scorer"
echo

# --- Merge back to main: runs and is verified -------------------------------
git switch -q main
git merge -q --no-ff add-scorer -m "Merge add-scorer"
echo "after merge, main has score.py: $([ -f loanpkg/score.py ] && echo yes || echo no)"
echo "main advanced: $([ "$(git rev-parse main)" != "$main_before" ] && echo yes || echo no)"

Merging to main is also the moment a release happens, and git has a dedicated tool for marking one: a tag. A branch pointer moves as you commit; a tag is a fixed name pinned to one commit forever — git tag v0.2.0 stamps “this exact commit is release 0.2.0,” and git push --tags publishes it. That is what “install version 0.2.0” actually resolves to: a tag, not a branch. It is also the missing piece from the last lesson — you wrote version = "0.1.0" in pyproject.toml and read what MAJOR.MINOR.PATCH means; tagging is where that number stops being a static label and gets bumped.

The bump is a decision you make at each release, and the rule is exactly the semver contract from before, now run in reverse — instead of reading a number, you choose the next one based on what changed since the last tag. Broke the interface (renamed a function, changed what an argument means)? MAJOR: 1.4.2 → 2.0.0. Added something new without breaking what was there? MINOR: 1.4.2 → 1.5.0. Fixed a bug and changed nothing about how the package is used? PATCH: 1.4.2 → 1.4.3. The discipline is that the bump is not a mood — it is a factual claim about compatibility, and the person who upgrades trusts it. Get it wrong (ship a breaking change as a PATCH) and you break their build on an upgrade they had every reason to think was safe. So the release ritual is small and fixed: merge the PR to main, decide the bump from what the diff actually did, set that version in pyproject.toml, commit it, and git tag the result. That is the loop you run every time you cut a release on any real package — and as the later modules change this one, each project names the bump that change would be (major, minor, or none), so you practice the judgment even where you are not cutting the tag yourself.

The output shows main unchanged while commits pile up on add-scorer, then advancing only when the branch merges back. The push and gh pr create lines are echoed (they need GitHub), but the local branch-and-merge is real, and it’s the part that proves main was never at risk.

left to right direction
[main @ c1] as m1
[main @ c2] as m2
[main @ merge] as m3
[feature @ f1] as f1
[feature @ f2] as f2
m1 --> m2
m2 --> m3 : merge (via PR)
m1 --> f1 : git checkout -b
f1 --> f2
f2 --> m3

The fork, the divergence, the rejoin: feature carries its own commits while main keeps working, and the PR is the gate where the two histories come back together, after review, not before.

# What happened (the broken way):
#   edit -> git add -> git commit (on main) -> git push   # main is now broken
#
# Rewrite as the branch-based flow that keeps main working. Name each command.
steps = ["???"]
print(steps)

steps = [
    "git checkout -b add-scorer   # isolated branch off main",
    "edit + git add + git commit  # these commits move ONLY add-scorer",
    "git push -u origin add-scorer",
    "gh pr create                 # open a PR; CI runs on the diff",
    "merge after review           # main moves forward only now",
]
for s in steps:
    print(s)

What you built

Your package now has a history and a home on GitHub, and you have the loop (edit, add, commit, push) plus the branch workflow that keeps main safe while you work. Every later module leans on this: the projects expect branches and PRs, and the CI module gates merges on exactly this flow.

• Git tracks work in three places: the working tree (files on disk), the staging area (git add chooses the next commit’s contents), and commit history (a parent-pointer chain you can walk back).
• git push is the separate step that sends local commits to the remote. Committed is not the same as pushed.
• A branch is a movable pointer to a commit; committing on a feature branch advances only that pointer, so main keeps working.
• A pull request wraps the branch’s diff for review and CI, and the merge moves main forward only when the change is ready.

Check your understanding:

• What is the difference between git add and git commit, and between git commit and git push?
• Why does committing on a feature branch leave main untouched?
• What does a pull request add beyond merging the branch yourself?