ML Explorer · Level 1 · Data & intuition
Data as numbers
A model only ever sees numbers. Each example becomes a short list of numbers, a vector, which is also a point you can draw and measure.
Overview
Imagine six pieces of fruit. Each one has been weighed and measured across, and the results written in a table. Before a model can learn anything from them, the fruit have to become numbers like these.
Once each fruit is a list of numbers, we can draw it as a point and measure how far apart two fruit are. Many methods use that distance to decide which examples are alike. There is a catch: the answer can depend on the units you measured in.
The fruit in this lesson are made up. See "About the data" at the end.
You'll learn
- what examples, features and labels are
- how one example becomes a vector, and a point on a graph
- how to measure the distance between two examples, and why units can change the answer
-
Rows and columns
Each row of the table is one fruit, called an example. Each column is one thing measured about it, called a feature.
There are six examples and two features: mass in grams, and diameter in centimetres. The diameter is the width across the widest part.
The kind column is different. It is the answer a model might later learn to predict from the measurements, like the mark in Lesson 1. A column like this is called the label. It is what a model would predict, so it is not one of the features.
The six fruit # kind mass (g) diameter (cm) 1 apple 140 7.0 2 orange 230 8.0 3 nectarine 136 6.4 4 plum 62 5.0 5 apple 152 7.1 6 apricot 40 4.4 The maths
We write for the number of examples and for the number of features. Here and .
The feature columns together form a grid of numbers with rows and columns. It is called the data matrix and written .
-
One row is a vector
Take one row and keep only its features. You get an ordered list of two numbers, called a vector, written x = (x₁, x₂).
Here x₁ is the mass and x₂ is the diameter. The order is fixed: mass always comes first, so every fruit's vector reads the same way.
Vectors are written in bold. So x stands for a whole row, while x₁ and x₂ are single numbers inside it.
Pick a fruit from the menu to see its vector.
The maths
For example number ,
The first subscript says which example, and the second says which feature. is the set of all pairs of real numbers.
With features, a vector has entries: .
-
A vector is a point
Two numbers give a place on a graph: go x₁ across and x₂ up. Each fruit becomes a point.
The table has turned into a picture. Across is mass, up is diameter, and each dot is one fruit, numbered as in the table.
The arrow runs from the corner, where both values are zero, to the chosen fruit. It is the same vector drawn as a movement. The dashed lines show its two parts.
Heavier fruit sit further right, and wider fruit higher up. In these fruit the two go together, so the dots run from bottom left to top right.
The maths
The arrow starts at the origin, . Its two parts are the short sides of a right-angled triangle, and the arrow is the long side. So, by Pythagoras' theorem, its length is
This length is called the norm of .
-
Distance between two fruit
Pick two fruit, a and b. The distance between their points measures how different they are.
The dashed sides are the gap in mass and the gap in diameter. The solid line is the long side of that right-angled triangle. Its length comes from Pythagoras: square both gaps, add them, and take the square root.
A small distance means two fruit have similar measurements. Many methods use this to find examples that are alike.
The two axes use different scales, so the length of the line on screen is not the distance. The number in the box is.
Notice that the distance adds a gap in grams to a gap in centimetres. Keep that in mind for the next step.
The maths
The distance between and is the norm of their difference, :
With features it is . This is the Euclidean distance. It is the same whichever fruit you call a, and it is zero only when the two vectors are equal.
-
Units change the answer
In the distance, a gap of 1 g counts as much as a gap of 1 cm. But 1 g is a tiny difference between two fruit, and 1 cm is a big one.
In grams and centimetres, mass decides almost everything. Fruit of about the same weight come out close, even when their sizes differ.
Standardising fixes this. For each feature, subtract its mean (the average) and divide by its standard deviation (a measure of how spread out the values are). Each feature is then counted in standard deviations from its average, and has no units, as long as the feature's values are not all the same.
Switch between Raw units and Standardised to compare. In the standardised view both axes use the same scale, so the length on screen does match the distance.
In raw units, the closest pair is . After standardising, it is .
Had we weighed in kilograms, the diameters would decide almost everything instead. Standardising gives the same result whatever units you start from.
Standardising is one common choice, not the only one. Which scale is right depends on the problem.
The maths
Write for the value of feature in example , as in step 2. Each feature has its own mean and standard deviation , taken over the examples. The standardised value, or z-score, is
where
This page uses the population standard deviation, which divides by . Lesson 3 explains standard deviation properly, and when to divide by instead. If a feature's values were all the same, would be 0 and the z-score would not be defined.
Afterwards each feature has mean 0 and population standard deviation 1. Changing units, say from grams to kilograms, multiplies every , and for that feature by the same number, so does not change.
For mass, = g and = g. For diameter, = cm and = cm.
Quick check
Lesson complete
- A data table has one row per example and one column per feature.
- Each example is a vector: its feature values in a fixed order. A vector is also a point.
- Distance compares examples, but it is only fair once the features are on comparable scales.
Common misconceptions
- "A vector has to be an arrow, as in physics." Here it is simply an ordered list of numbers. The arrow is one way to draw it.
- "The length on the graph is the distance." Only when both axes use the same scale.
- "Numbers are numbers, so units don't matter." Switching from grams to kilograms can change which examples count as close.
- "Standardising throws information away." It shifts and rescales each feature. The values keep their order and their relative gaps within each feature. Keep each feature's mean and standard deviation and you can undo it.
Further reading
- Deisenroth, Faisal & Ong, Mathematics for Machine Learning, Section 8.1.1, "Data as Vectors", and Section 3.3, "Lengths and Distances".
- James, Witten, Hastie & Tibshirani, An Introduction to Statistical Learning (2nd ed.), Section 4.7.6, "K-Nearest Neighbors", which explains why the scale of the variables matters for distances.
About the data. The six fruit are made up. Their masses and diameters are realistic but invented for this lesson. They are fixed values written into the page, not random draws, so there is no seed. Treated as a sphere, each fruit's mass and diameter give a density between about 0.8 and 1 gram per cubic centimetre, close to that of water, so the two columns agree with each other.
If this helped, you can .