First Step: the Requirements
Most projects that land on my desk with some version of “can we use machine learning for this?” do not start as machine learning projects.
Sometimes they need an analysis. Sometimes a rule-based system is enough. Sometimes it is a small script running on a schedule.
And often, that simpler version is exactly the right MVP. It lets you find out whether the idea is useful, what data you actually need, and where the simple approach starts to break before you invest in a model.
Figuring that out is part of the job.
So this lesson starts by screening the request. If machine learning still makes sense after that, we write docs/requirements.md: what we are building, why we are building it, what success looks like, who owns which decisions, and what is deliberately out of scope.
We are not designing the system yet. First, we need to agree on the problem.
This Lesson’s Goals
This lesson produces one file, docs/requirements.md — the document that gets everybody to commit to what is being built, written so the people paying for it can read every line.
By the end, you should be able to:
-
Screen a request before you scope it, and distinguish between an analysis, a rule, a small piece of software, and a problem that actually needs a model.
-
State what the business is trying to change, rather than what the model predicts. They are different sentences, and only one of them is the product.
-
Write down what you are deliberately not doing, which is the section that prevents half the arguments you would otherwise have in month three.
Follow Along
Every lesson in this course works in one repository, and each lesson tells you where to find the files used in the examples.
These commands need the course CLI, installed in How the Work Works. If dutchengineer is not a command your shell knows yet, start there — it is two lines and a sign-in.
$ dutchengineer get booking_scorer
$ cd booking_scorer
$ dutchengineer data pullget clones the project as Sam left it: a notebook, a data-pull script, a dependency list, and not much else.
data pull fetches the two licensed data extracts and verifies their checksums. The repository contains the means of getting the data rather than the data itself.
Each example in this course says how to run it and what it should print. dutchengineer setup writes the data paths into your shell profile when it installs, so a fresh terminal already knows where everything is. If an example cannot find the data, running dutchengineer setup again is the fix.
The file we write in this lesson goes here:
booking_scorer/docs/requirements.mdDoes This Even Require Machine Learning?
Here is the request as it arrived.
Jordan, who runs Sales and is acting as product manager on this project, wants a retention coupon shown during checkout when a booking looks likely to cancel.
There are several different projects hiding inside that request, and only one of them requires machine learning.
Jordan knows the outcome she wants: fewer cancellations without giving discounts to guests who were going to stay anyway.
Choosing the simplest thing that can get us there is our job.
So before we touch the model, here are the different paths we could take with this project.
Path 1: Test the Idea Without Predicting Anything
We may not need to predict anything yet.
Maybe we show a coupon when a guest looks like they are leaving the checkout page. That is front-end logic, not machine learning, and we could ship it quickly.
Jordan did not ask for this version, but it answers an important question underneath her request:
Do coupons actually change anybody’s mind?
That makes this a useful MVP. If coupons do not reduce cancellations, there is little value in building a sophisticated system for deciding who gets one.
Path 2: Use Something We Already Know
Sometimes the answer already exists in the data.
If Jordan wants to show the coupon to guests who booked a refundable rate, that information is already part of the booking.
Read the value. Apply the condition.
There is nothing to predict.
Path 3: Use a Simple Rule
Sometimes one or two variables get us most of the way there.
Suppose bookings made more than ninety days in advance are much more likely to cancel. We could test:
Flag every booking made more than ninety days out.
If that solves enough of the problem, the rule may be the product.
And even if it does not, it gives us something valuable: a baseline.
If we eventually build a model, the model needs to improve enough over this simple version to justify everything that comes with operating one.
Path 4: Build a Model
Now suppose the simple rule does not hold.
Cancellation risk depends on lead time, price, rate type, previous stays, what is happening in the destination that week, and how those things interact.
A long lead time might mean one thing for a cheap flexible rate and something completely different for a prepaid suite. Those relationships can change over time too.
At that point, turning the decision into a growing pile of if statements stops making sense.
Now we have a reason to consider a model.
The important part is that all four are legitimate outcomes of the initial conversation.
We are not trying to prove that Jordan’s request deserves machine learning. We are trying to find the simplest version of the product that can produce the business outcome she wants.
For this course, we are going to continue down Path 4. We have historical booking data, cancellation outcomes, and enough interacting factors that a single rule is unlikely to describe the behavior well.
But we are keeping Path 3 around.
Before this model earns its place in production, we need to measure a simple baseline on the same data and show that the model is meaningfully better. We do not know that number yet, so it goes into the requirements as an open question rather than something we invent.
That is enough to move forward.
Now we can write the requirements.
What Goes in the Document
docs/requirements.md has seven parts, and the whole document is written so somebody without an engineering title can read every line of it.
Jordan reads it. Elena, who owns the budget this coupon comes out of, reads it. If a line needs an engineer to translate it, it probably belongs in the system design instead.
# Title
## Context
[What already exists and anything from before this project
that affects where we are starting.]
## Goal
[What the stakeholder is actually trying to change.]
## Stakeholders and Ownership
[Who is involved and which decisions each person owns.]
## Requirements
[The specific things the product has to do.]
## Out of Scope
[What we are deliberately not doing.]
## Deadlines
[The current estimate and what could move it.]
## Related Links
[The system design, notebook, and anything else worth
keeping with the project.]Now we can take those sections one at a time.
Context
Context is what already exists and what we are walking into.
It is easy to skip because it feels like background rather than part of the work. But six months from now, one of the most useful questions this document can answer is: why did we make the choices we made?
Half of that answer is usually what was already true when the project started.
On this project, there is more context than usual because we are not starting from nothing:
Context
Sam (Data Science) built a cancellation model in a notebook before this project existed, trained on a 2015–2017 extract of two properties. The notebook reports 0.735 accuracy on her own split.
It has never run anywhere but her laptop, nothing calls it, and Sam has moved to pricing, so the notebook is currently the only description of the model that exists.
Nothing about this has been verified yet. That number is what the notebook says, not what we have reproduced.
That last line is doing more work than it looks like.
Someone telling you a number is not the same as you having checked it.
The moment you write 0.735 into a document without that caveat, it becomes a fact everyone downstream starts planning around.
Review the Notebook is where we find out whether it holds up. Until then, it goes into the requirements with its provenance attached.
Goal
The goal answers what the stakeholder actually wants to change.
By the time you are writing it, you have usually already had the conversation. The point is to capture it so that four months from now you can ask: what did they originally want, and where did we end up going?
Goal
Jordan (Sales) wants guests who look likely to cancel a booking to be offered a retention coupon while they are still completing the booking, so that fewer confirmed bookings turn into cancellations.
Success is measured as a reduction in the cancellation rate on bookings that were offered a coupon, against comparable bookings that were not.
That second sentence is the important one.
The measure of success is a business number, not a model number.
Nobody funding this project cares whether the model goes from 0.735 to 0.742 unless something on their side moves with it. That gap is how ML projects quietly lose support: the model metric improves while the business metric stays exactly where it was, and the person paying for it has no way to tell the difference between a better model and a busier team.
Sam’s 0.735 matters, and we will use it.
It is not what Jordan is buying.
And if you cannot get a business measure out of the conversation yet, write down that you could not and who owns the answer.
An unanswered question with a name on it is a working document.
A guess is a trap.
Stakeholders and Ownership
Machine learning projects rarely have one person who can answer every question.
You end up talking to the data team, the data scientist who built the model, the software engineers who own the application you are integrating into, other ML engineers, the product manager, Finance, and sometimes a project manager keeping the whole thing moving.
Keeping all of those people aligned is a large part of what this document is for.
The trap I fell into early, more than once, was treating the most enthusiastic stakeholder as the source of truth.
Jordan asked for this. Jordan is acting as product manager. So Jordan confirms everything.
Except Jordan cannot confirm most of it.
In the document, that becomes a table because a table is easy to scan and easy to correct:
Stakeholders and Ownership
Who Owns Confirms Jordan (Sales) The product decision Coupon rule, what the guest sees, how success is measured Sam (Data Science) The model as built What it predicts, what the 2015–2017 extract covers Priya (Data Engineering) Production data sources Whether each feature is available at request time Marcus (Website) Checkout Wait budget, peak bookings/day Elena (Finance) Coupon budget Budget ceiling, what the audit record must contain Nadia (ML Engineering) Code review Whether what we built is fit to run, and what changes first Sarah (ML Engineering) The service and this document Delivery estimate, keeping the above current
And it will change.
On a project of any size, ownership moves. People switch teams. Someone goes on leave. Sam has already moved to pricing, and that is a fact about this project rather than a hypothetical.
Keep the table current.
A stale owner column is worse than no owner column because somebody will act on it.
Requirements
This is where the specific things the product has to do get written down.
Some of them will come directly from the stakeholder. Some will come from conversations with the other teams. Some are things you noticed while working through the problem that have nowhere else to live yet.
That is fine. They go here.
For the cancellation scorer:
Requirements
When a guest completes a booking, the system produces a cancellation likelihood for that booking.
Scoring happens once, at booking completion. Nothing goes back and re-scores an open booking.
The rule that turns a likelihood into a coupon lives in the product, not in the model. Jordan owns the cut-off and can change it without an engineering release.
The guest sees the coupon while they are still on the checkout page, or they see nothing.
If the scorer is slow or unavailable, the booking completes normally with no coupon shown. Losing a booking to a coupon system is never an acceptable trade.
Every offer that gets issued is recorded somewhere Elena can query, with enough detail to reconcile against the budget.
Total coupon spend stays under a ceiling Elena sets. Open — Elena owns the number.
Checkout can wait roughly one second for an answer. Proposed — Marcus confirms.
Peak volume, in bookings per day, sizes everything downstream. Open — Marcus owns the number.
Before we commit to the model, we record what a simple rule achieves on the same bookings, so we know what the model has to beat. Open — Sarah owns it, and it stays open until somebody measures it.
Jordan’s hunch that “our regulars” are the ones to target is written here as a hypothesis, not a requirement. It gets tested against the data before anything is built around it.
The scorer reads booking details and writes a record of every offer. Who may call it, and who may read those records, is decided before it is reachable by anything but us. Open — Marcus and Elena.
Several of those are open questions with names attached instead of answers.
That is what the document should look like at this stage.
I would rather hand Jordan a document with clearly named open questions than one that looks finished and is quietly wrong in four places.
Number eleven is worth sitting with because it is one of the easiest ways for a project to go sideways.
Jordan’s hunch might be right. Repeat guests might actually be the group worth targeting.
But it arrived as a feeling.
If we write it down as a requirement, we start building around it before we know whether it is true. If we label it as a hypothesis, the notebook review can test it.
You do not have to argue with the hunch.
You just have to put it in the place where it gets tested.
Number twelve is another one that is easy to leave out.
Every system that holds data eventually has an answer to two questions:
Who may call this?
Who may read what it stores?
The only question is whether those answers were chosen deliberately or inherited from whatever was easiest to configure when the system shipped.
We are not doing a full threat model here. That belongs elsewhere.
We are simply making sure the people who actually own those decisions answer them while changing the answer is still cheap.
Out of Scope
Out of scope might be the most useful section in the entire document.
Especially the first few times you take a product from an idea to a production ML system, this is where a lot of the pushback happens.
One person says this is the important part.
The next person says no, this is the important part.
Both of them may be right about their own job.
The only way through that is to decide what belongs in this version.
Out of Scope
Re-scoring a booking after it is completed. One score, at completion, and that is what this version does.
Anything on the mobile app. Checkout on the website only.
Choosing the coupon amount with the model. The model gives a likelihood; the amount is a business decision.
Retraining on a schedule. The first release ships one model. How and when it gets refreshed is a follow-up.
Targeting rules built on Jordan’s “regulars” hunch until the data supports it.
There is another reason I like writing this section early.
If the project succeeds, this list becomes the beginning of the roadmap.
Scheduled retraining becomes another piece of work. Mobile becomes another piece of work. Re-scoring bookings becomes another conversation.
You wrote down the possible v2 work while protecting v1 from it.
Deadlines
What do the deadlines look like?
They are estimates. They will change.
Give them anyway.
Deadlines
Requirements and system design agreed: end of week 2.
A service that returns a score in staging, callable by checkout: end of week 6.
Checkout integration and a first live cohort: end of week 9, dependent on Marcus’s team’s release calendar.
Estimates, not commitments. The week 9 date moves if the week 6 date moves.
Having that conversation before you start takes a surprising amount of pain out of the later part of the project, when you are heads-down building and somebody asks where it is.
The value is not that the estimate turns out to be perfect.
It is that everybody already knows the shape of the work.
When something moves, it becomes a conversation instead of a surprise.
And you do not need false precision to be useful.
Weeks are fine.
“End of quarter, assuming X and Y happen first” is fine.
Related Links
The last section is the loosest.
What else should somebody working on this project be able to find from here?
Related Links
System design — how the system works and why. Does not exist yet; linked when it does.
Sam’s notebook and the 2015–2017 data extract. The only description of the model that currently exists.
Booking cancellation data parsing notes from last year’s analysis — context for what the columns mean.
The system design is one you will almost always have because we still need to work out what this looks like in production.
But this section does not have to be purely technical.
Maybe somebody did a data-parsing exercise a year ago that explains why a column has a bizarre name.
Maybe there is an old analysis everyone keeps referring to.
Maybe there is a notebook somebody else deployed.
If the next person would waste a day rediscovering it, link it.
Summary
docs/requirements.md should answer:
- Does this problem actually need a model, or is there a simpler version we should try first?
- What does the person asking actually want to change?
- How will we know whether the product worked?
- Who owns each decision, and which answers are still open?
- What must this version of the product do?
- What are we deliberately not doing?
- When do we currently think it lands, and what could move that date?
- Where can the next person find everything else?
Now Build It Yourself
Same document, different product: a hospital readmission scorer, where a care coordinator stays responsible for the decision and one of the owners is clinical governance.
The requirements document now goes out to the people whose names are on it.
That review is its own stretch of time, and it ends in an agreement that will not satisfy everybody.
Next, we open the notebook Sam left behind.
Before we can decide what the production system should look like, we need to know what has already been built — and whether the number in the notebook actually holds up.