Make the Imports Robust

I once inherited a folder of scripts where every file appended to sys.path to find its neighbors. It worked. Right up until I moved one file, and then several others broke, and importing any single file quietly ran all of them because the work happened at import time. That folder had simply never been moved before. The first time I tried to reuse one function from it, I triggered three unrelated jobs and spent an afternoon figuring out why. Working, I learned that day, is not the same as robust.

Here’s the part that got me. Nothing in that story was a code bug. The code did not change between “works on my laptop” and “fails under the scheduler.” What changed was how it was launched, and that alone is enough to make the exact same import line succeed in one place and raise ModuleNotFoundError in another. The package you built earlier in this module has that same latent fragility. It ran fine for you, then went under version control in Lesson 3, and the moment it was shareable it started moving: a teammate clones it, a scheduler runs it from a different directory, a test runner imports it from somewhere else, and it breaks three different ways. Putting it on GitHub is exactly what exposes this, because code that only ran from one folder now runs from everywhere.

So in this lesson we will work out the two invisible mechanisms behind that afternoon:

1. How Python actually decides where an import resolves.
2. What actually runs the moment you import a file.

Get those two right and your package runs the same from anywhere. Get them wrong and it breaks the first time someone else clones it.

Why imports break when you move the code

The rule that explains the whole mess is short: imports resolve against sys.path, not against the directory you happened to run from. Your package worked earlier because you ran it one specific way. It breaks the moment something runs it a different way: moved to a new folder, scheduled by a different tool, collected by a test runner from elsewhere.

Let me walk what actually happens on import foo. Python first checks sys.modules, its cache of already-imported modules. On a miss, it hands the name to a series of finders, and the finder for ordinary files walks the entries of sys.path in order, stopping at the first directory that contains a matching module. Here is the catch. sys.path is just a list of strings, and its first entry depends entirely on how you launched the program:

• python script.py puts the script’s own directory first.
• python -m pkg.script puts the current working directory first.
• an interactive session puts '' (the cwd) first.

Those are different directories. So the identical import helpers line resolves to a real file under one launch and finds nothing under another, not because anything in the code moved, but because entry zero of the search path moved. That is the entire failure: a cwd-dependent first entry that quietly changes underneath you.

Here is the exact procedure Python runs on import foo, so the precedence is unambiguous:

1. Check sys.modules. If foo is already there, return the cached module and stop. The import body never runs a second time.
2. On a miss, try each finder in sys.meta_path in order.
3. The path-based finder scans sys.path left to right. The first directory that contains a matching module wins, and the remaining entries are never consulted.
4. sys.path[0] is set by the launch method (the three cases above); the rest of the list is the installed environment.

Step 3 is the one that explains shadowing. Because the first match ends the search, a module sitting in sys.path[0] hides any same-named module installed later in the list. A file named random.py in your launch directory will be imported in place of the standard library’s random, and nothing warns you. The search simply stopped at entry zero and never reached the real one.

The editable install from Lesson 1 is what sidesteps it. Installing the package registers it on a path that is present regardless of your working directory, so import mypkg resolves through the installed pointer instead of through whichever folder you launched from. That is why I keep calling the install load-bearing rather than cosmetic: it moves resolution off the fragile, cwd-dependent sys.path[0] and onto a stable one. The fix for a flaky import is never another sys.path.append. It is the package plus python -m.

The shadowing above sounds like something you would notice, but it has a quieter everyday form worth a warning: naming one of your own files after something you import. Save a random.py or an email.py next to your code and import random loads your file instead of the standard library’s, then raises a baffling AttributeError when the function you expected is not in it. Same first-match-wins rule, now firing on a name collision you created by accident. When any import misbehaves like this, when the wrong version runs or a name resolves to something unexpected, there is one diagnostic that ends the guesswork instantly: ask the module where it came from. python -c "import mypkg; print(mypkg.__file__)" prints the actual file Python loaded, and that single path settles which copy won the search. If it points into your src/ tree, the editable install is working. If it points at a same-named file in your launch directory, a stale copy in site-packages, or the wrong virtual environment, you have found the shadow without theorizing about it. The fix for the collision is short: do not name your modules after packages you import. And __file__ is how you confirm which file actually won when you are not sure.

Print the moving part directly and the bet a bare import is making becomes visible. Entry zero of the search path, whatever your launch put there:

"""Lesson 3.1 Show — why imports resolve differently when you move the code.

Python resolves a bare import against `sys.path`, and `sys.path[0]` is set from how
the program was launched (the script's directory when run as a file, or the current
directory under `python -m`). That is why `import data` works from one location and
breaks from another: the resolution context moved. The fix the lesson builds toward
is an installed package, so imports resolve by package name regardless of where you
launched from — not by poking `sys.path`, which is the fragile anti-pattern.
"""

import sys

# sys.path[0] is the launch-dependent entry that makes bare imports brittle.
print(f"sys.path[0] (launch-dependent): {sys.path[0]!r}")

# A bare, path-relative import works only when sys.path happens to include the right
# directory. Demonstrate both outcomes without crashing the example.
try:
    import this_module_does_not_exist_relpath  # noqa: F401

    print("relative import resolved (sys.path happened to include its dir)")
except ModuleNotFoundError:
    print("bare import FAILED: not on sys.path from this launch context")

# An installed/stdlib package resolves by name regardless of launch directory.
import json  # noqa: E402

print(f"installed-package import resolves anywhere: json from {json.__name__!r}")

Printing sys.path[0] makes the moving part visible: it is whatever your launch method put there, and a bare relative import is betting its life on that entry happening to contain the module. An installed package does not take that bet. It resolves by name no matter where you are.

The example above can only run one way at a time, so it cannot show the move itself. This is what the move looks like: the same file, the same import helpers line, launched two ways, with sys.path[0] printed each time. One resolves, the other raises ModuleNotFoundError, and the code never changed.

# project/pkg/app.py
import sys
print(f"sys.path[0] = {sys.path[0]!r}")
import helpers   # a sibling module: project/pkg/helpers.py


# --- Launch A: run the file directly, from inside project/pkg/ ---
#   $ cd project/pkg && python app.py
#   sys.path[0] = '/abs/project/pkg'      <- the script's OWN directory
#   (import helpers SUCCEEDS: helpers.py is in sys.path[0])
#
# --- Launch B: run it as a module, from the project root ---
#   $ cd project && python -m pkg.app
#   sys.path[0] = '/abs/project'          <- the CURRENT directory, not pkg/
#   ModuleNotFoundError: No module named 'helpers'
#   (helpers.py is in project/pkg/, which is no longer entry zero)

Nothing in app.py differs between the two runs. Entry zero moved from project/pkg (the script’s directory) to project (the cwd), and the bare import helpers that resolved against the first finds nothing against the second. That is the failure the install removes. With the package installed, import pkg.helpers resolves through the install regardless of which directory is entry zero.

Here is the resolution walked one step at a time, so the ordered search and its launch-dependent first entry are visible.

# flat folder:
#   project/
#     train.py     # does: import helpers
#     score.py     # does: import helpers
#     helpers.py
# Works from inside project/, breaks from anywhere else (sys.path[0] changes).
#
# Sketch the package layout + the install that fixes it.
layout = "???"
print(layout)

layout = """
project/
  pyproject.toml
  src/
    mypkg/
      __init__.py
      helpers.py
      train.py    # from mypkg import helpers   (resolves by package name)
      score.py
# then: uv pip install -e .
# Now imports resolve by PACKAGE NAME through the installed pointer, not by
# cwd, so train and score find helpers from any directory.
"""
print(layout)

Why importing should not run the job

You already added the if __name__ == "__main__": guard in Lesson 1 and saw that it works. Now the why, because the failure it prevents is one of the nastier ones in ML. Importing your training module, just to borrow one function, kicks off the entire training run.

Here is the mechanism. The first time a module is imported, Python executes every top-level statement in it, top to bottom, and “top-level statement” includes assignments and function calls, not only def and class. As it runs each line it binds that name on the module object, and when the body finishes it stores the completed module in sys.modules. Every later import of that name returns the cached object without re-running the body. That is the asymmetry that catches people: top-level code runs exactly once, on first import, but a def or class only binds a name. It defines. It does not call.

So where does the guard fit? Python sets a module’s __name__ to the string "__main__" only when that file is the program being run directly, and to the module’s dotted import name ("mypkg.train") when it is imported. if __name__ == "__main__": is just a runtime check that is true only in the direct-run case. It does not suppress top-level execution in general. It gates one block on which case you are in. The discipline that falls out is simple and load-bearing. Definitions go at the top level, where importing them is free because it only binds names. The “actually do the run” calls go inside the guard, where they fire only when the file is the program.

Watch the split in action, the top-level marker firing on any touch, the guarded one only on a direct run:

"""Lesson 3.2 Show — importing your module should not run your job.

Top-level code runs the moment a module is imported; code under
`if __name__ == "__main__":` runs only when the file is executed directly. If the
training job lives at top level, then `import features` (to reuse one function)
silently kicks off the whole job. The marker pattern below makes the difference
observable: the top-level marker always fires, the guarded marker only on direct run.
"""

# Top-level: fires on BOTH import and direct run — anything expensive here is a trap.
print("TOP-LEVEL marker fired (this runs even on `import`)")


def real_work() -> str:
    return "the job ran"


if __name__ == "__main__":
    # Guarded: fires ONLY on direct run / `python -m`, never on import.
    print("GUARDED marker fired (direct run only)")
    print(real_work())

The top-level marker fires whenever the module is touched, including a bare import. The guarded marker fires only on a direct run. Picture the version without the guard: a train.py that calls train() and saves the model artifact at the top level will retrain and overwrite the live model the instant anything imports it. No output, no warning. Months later you import that file from a notebook to reuse its train() helper, and you have silently clobbered production. The guard is what makes a file safe to import for its parts without triggering its job.

There is a sharper version of this same trap that the __name__ mechanism creates directly, and it is worth naming because it produces a bug that looks impossible: a module that runs twice in one process. When you launch python -m mypkg.train, that file runs under the name __main__. If any other module in the same run then does import mypkg.train, to reuse a function from it, Python does not recognize it as the file already running, because it is cached under __main__, not mypkg.train. So it loads and executes the file a second time, under its real name, in a separate module object. Now every top-level effect in that file has happened twice: a config parsed twice, a model loaded into memory twice, a registry or a counter initialized twice, two copies of what you assumed was one object. The symptom is baffling. Duplicated log lines, a “singleton” that is not, memory that is double what it should be, and nothing in the code looks wrong, because the code is fine. It simply ran twice under two identities. The rule that avoids it is the same discipline the guard already enforces, taken one step further: the file you run as __main__ should be a thin entry point that imports the real work from another module and calls it, never the module other code imports for its functions. Keep the logic in mypkg.train and let mypkg.__main__ be the tiny thing that calls it, and there is only ever one copy of the logic, imported by one name, no matter how it is launched.

def feature_names() -> list[str]:
    return ["loan_amnt", "annual_inc"]


def train_and_save(path: str) -> None:
    # Pretend this fits a model and writes it. The print stands in for the file write.
    print(f"OVERWROTE {path} with a freshly trained model")


# Top level: runs on import too. Predict what `import train_job` does here.
train_and_save("model.pkl")

def feature_names() -> list[str]:
    return ["loan_amnt", "annual_inc"]


def train_and_save(path: str) -> None:
    print(f"OVERWROTE {path} with a freshly trained model")


if __name__ == "__main__":
    # Fires only on a direct run / `python -m`, never when a notebook imports
    # this module for feature_names(). The production model is safe from the import.
    train_and_save("model.pkl")

One last import-time trap hides where it does not look like state at all: a function’s default arguments. Python evaluates the defaults on a def line once, when the module is imported, not on each call. def add_row(row, batch=[]) therefore creates a single list at import time, and every call that omits batch appends to that same shared object. The function looks stateless while carrying a module-level accumulator disguised as a parameter, and the symptom is the classic one: results that depend on how many times the function has been called before. The fix is the same move as the __main__ guard, deferring the work past import. Default to None and create the fresh value inside the body (if batch is None: batch = []), so it is built at call time, once per call. The rule of thumb: an immutable default (a number, a string, None, a tuple) is safe. A mutable one (a list, a dict, a set) is import-time state shared across every call that omits the argument.

What you hardened

The package did not change shape this lesson. You learned what was holding it up. Imports resolve against sys.path, whose first entry shifts with how you launch, which is why the install and python -m are what make resolution stable instead of luck. And top-level code runs on import, which is why the __main__ guard is the difference between a file you can safely import and one that detonates its job when touched.

• Imports resolve against sys.path; its first entry depends on the launch method, so the same import can succeed one way and raise ModuleNotFoundError another.
• The editable install moves resolution onto a stable, name-based path: the real fix for a fragile import, never sys.path.append.
• First import runs all top-level code once and caches the module in sys.modules; a def/class only binds a name.
• if __name__ == "__main__": gates one block on direct-run vs import; keep run-calls inside it, definitions outside.
• A mutable default argument is import-time state in disguise: defaults are evaluated once at def time, so batch=[] is one list shared across every call. Default to None and build the value inside the body.

Check your understanding:

• Why does launching from a different directory change which imports succeed, and what does python -m plus an installed package fix?
• What value does __name__ hold when a file is imported, versus run directly?
• Does the guard stop all top-level code, or only the guarded block?