ML Explorer · Level 0 · Start here
What is a model?
A model is a rule learned from examples. Once it has learned, it can make a guess about a case it has never seen.
Overview
Suppose you have the records of 20 students: how many hours each one studied for an exam, and the mark each one got. A new student tells you they studied for six hours. What mark would you guess?
You would probably look at the students who studied about that long and guess something close to their marks. A machine learning model does the same job, but it writes the idea down as a rule with a few numbers in it. Those numbers are chosen by looking at the examples. Choosing them is called training.
The students in this lesson are made up, so we know exactly how their marks were produced. See "About the data" at the end.
You'll learn
- what a model is, and what its parameters are
- what it means to train a model on examples
- why a good model's predictions are still not exact
-
Start with examples
Each dot is one student. Across is the hours they studied; up is the mark they got, out of 100.
These 20 dots are the examples the model will learn from. Each example has an input, the hours, and a known answer, the mark.
Look at the overall shape. Students who studied longer tend to have higher marks, but the dots do not sit on a neat line.
The maths
We write the examples as pairs for . Here is student 's hours, is their mark, and .
The input is also called a feature, and the answer the target.
-
A rule is a guess
A model is a rule that turns hours into a guessed mark. This rule is a straight line: start at a, then add b marks for every hour studied.
The line is the rule. The mark it guesses is written ŷ, said "y-hat". Move the sliders: a sets where the line starts at zero hours, and b sets how steep it is.
The short vertical lines show how far each real mark is from the guess. The "average squared miss" on the stage multiplies each of those gaps by itself and takes the average. Smaller is better. Lesson 4 explains why we square.
The starting line is deliberately poor: it says that more study means a lower mark.
The maths
The numbers (the intercept) and (the slope) are the rule's parameters, collected in (theta). The semicolon separates the input from the parameters.
The average squared miss is the mean squared error:
-
Training picks the best rule
Learning means choosing a and b so that the line fits the examples as well as possible.
The line has moved from your guess to the best fit. Training looks for the a and b that make the average squared miss as small as it can be. For a straight line there is exactly one best answer (as long as the hours are not all the same), and it can be worked out with a formula.
The dashed line is your guess from the last step, kept for comparison.
"Best" here means the smallest average squared miss. Other ways of measuring a miss can give different lines.
The maths
Training solves
This is least squares. For one input it has a closed-form answer, where and are the means of the hours and the marks:
It needs the hours to vary: if every student studied the same time, the formula would divide zero by zero, and many lines would tie for best. A hat marks an estimate: and are learned from data, and is the rule's guess. Lesson 6 derives the formula.
On these examples, = and = .
-
Predict a new case
Now use the rule. Drag the ring, or move the hours slider, to choose a new student's study time and read off the predicted mark.
The new student is not one of the 20 examples. The model has never seen their mark, and it does not need to: it puts their hours into the rule and reads the guess off the line. This is prediction.
Training happened once, using the examples' hours and marks. Prediction can happen as often as you like, and needs only the hours.
The slider stops at 1 and 10 hours, about the range the examples cover. Outside that range the line still gives an answer, but no data backs it up.
The maths
Training uses inputs and answers, , to fix . Prediction uses and a new input only.
-
Useful, not perfect
The line gives a sensible guess, but real marks scatter around it. Some students did better than the rule predicts, and some did worse.
The vertical lines are the misses left after training. In real data, misses come from everything the rule leaves out: how well someone slept, what they already knew, luck on the day. In these made-up data, the random noise we added plays that part (see About the data).
A model simplifies on purpose. It keeps the main pattern and ignores the rest. The fitted line is itself an estimate, so it differs a little from the rule used to make the data.
So a prediction for a new student is a best guess, not a promise.
One caution: a pattern in data does not, on its own, show cause and effect.
The maths
Each miss is a residual, . A common way to write the whole story is
where (epsilon), often called noise, stands for everything the rule does not capture.
These examples were made with and plus random noise. Training on noisy examples found = and = : close, but not exact. With more examples the estimates tend to land closer.
Quick check
Lesson complete
- A model is a rule with some numbers left open. Those numbers are its parameters.
- Training chooses the parameters that make the rule fit the examples best.
- Prediction runs the trained rule on a new input. The answer is useful, not exact.
Common misconceptions
- "The model stores the examples." This one keeps only two numbers, a and b. The 20 students are not inside it.
- "A good fit proves cause and effect." It shows that two things go together in the data, nothing more.
- "A trained model can be trusted for any input." Far outside the range of its examples, it has no data to go on.
- "If the predictions miss, the model is broken." Some miss is expected, because the rule leaves out things that matter.
Further reading
- James, Witten, Hastie & Tibshirani, An Introduction to Statistical Learning (2nd ed.), Section 2.1, "What Is Statistical Learning?", and Section 3.1, "Simple Linear Regression".
- Deisenroth, Faisal & Ong, Mathematics for Machine Learning, Chapter 8, "When Models Meet Data".
About the data. The 20 students are made up. They are generated in your browser with the mulberry32 random number generator, seed 7. For each student in turn, the hours x are drawn uniformly between 1 and 10, and then the mark is y = 35 + 5x + 6z, where z is a standard normal random number. Marks are clipped to the range 0 to 100. The same seed gives the same students every time.
If this helped, you can .