Is a straight line appropriate?: Year 11 Extended sampling, correlation and regression

Warm-up

Three quick ones from earlier work. Try each in your head or on paper before you open the answer.

A line is y = 3x + 2. A dot is at (4, 17). Is the dot above or below the line, and by how much?
Answer
The line gives 14, so the dot is 3 above the line.
From Lesson 4: what does the regression line make as small as possible?
Answer
The total of the squared vertical gaps from the dots to the line.
From Year 11 Standard: a line fitted to data with x from 5 to 20 is used at x = 12 and at x = 60. Which is interpolation?
Answer
x = 12 (inside the data); x = 60 is extrapolation.

Words used in this lesson

residual: a dot's y minus the line's y at the same x. For example: a plant 3 cm above the line has residual 3.
model: a rule or equation used to describe or predict real data. For example: the regression line used to forecast sales.
appropriate: suitable for the data and the purpose. For example: a straight line is appropriate when the residuals have no pattern.
systematic: following a pattern rather than being random. For example: residuals that are positive, then negative, then positive.
extrapolation: predicting for an x outside the range of the data. For example: forecasting sales beyond the months measured.

Two sets of dots

A regression line can be worked out for any dots. Here are two sets of eight dots at x = 1 to 8. In one the dots form a rough band that rises; in the other they follow a clear curve that bends upwards. The residual of a dot is its gap from the line: the dot's y minus the line's y. It is positive above the line and negative below it.

Predict first: which set do you think will have the higher r, the rough band or the curve? And for which set does a straight line deserve to be called a good model? Slide through the dots and switch sets.

Dot at x 4

0.93r, to two decimal places

y = 2x + 3regression line

3residual of the chosen dot

Roughly straight data: r = 0.93 and the line is y = 2x + 3. The dot at x = 4 is 3 above the line.

The figure needs JavaScript. In words: eight dots at x = 1 to 8. The roughly straight set has heights 6, 5, 8, 14, 11, 16, 19, 17, with line y = 2x + 3 and r = 0.93; its residuals are 1, −2, −1, 3, −2, 1, 2, −2, with no shape. The curved set has heights 3, 6, 11, 18, 27, 38, 51, 66, with line y = 9x − 13 and r = 0.98; its residuals are 7, 1, −3, −5, −5, −3, 1, 7, positive at both ends and negative in the middle. The curved set has the higher r and the worse straight-line model.

Show me the steps

Roughly straight data, dot at x = 4: r = 0.93 and the line is y = 2x + 3. The dot is at 14 and the line gives 11, so its residual is 14 − 11 = 3.
The eight residuals are 1, −2, −1, 3, −2, 1, 2 and −2: some above, some below, no shape. Their total is 0. A straight line is a fair model.
Curved data: r = 0.98, higher than before, and the line is y = 9x − 13. Its residuals are 7, 1, −3, −5, −5, −3, 1 and 7.
The curved residuals are positive at both ends and negative in the middle, a U. The line misses the bend, so a straight line is the wrong shape here, whatever r says.

Judge a straight line by its residuals as well as by r.

The residuals of a regression line add to 0. Look at them in order of x, not just at their size.
A shapeless scatter of residuals about 0 supports a linear model. A U or an arch says the line is the wrong shape. Residuals that grow in size as x rises say the line's predictions are less reliable where the spread is large.
r measures how closely dots follow a straight line. r is a number about the dots you have, not a promise about dots you do not.
Predicting y from x inside the data (interpolation) is reasonable with a sound model. Predicting outside it (extrapolation) assumes the pattern goes on, and the cost grows with the distance.

A high r and a good model are different things. It is tempting to report r and stop. Two checks are needed: the residuals for the shape of the model, and the position of x for the trust in a prediction. A good report says both: “the line is a fair model for x from 1 to 8, and predictions inside that range are reasonable; beyond 8 it is untested.”

Worked examples

A line is y = 2.5x + 4. The dots (2, 11), (4, 13) and (6, 19) are three of the data. Find the three residuals.

The line gives 9, 14 and 19 at x = 2, 4 and 6.
Residual = dot minus line: 11 − 9 = 2, 13 − 14 = −1, 19 − 19 = 0.
The first dot is above the line, the second below it, and the third lies on it.

The residuals of a regression line at x = 1 to 7 are +4, 0, −2, −4, −2, 0 and +4. Check that they add to 0, and say what they suggest.

Their total is 4 + 0 − 2 − 4 − 2 + 0 + 4 = 0, as the residuals of a regression line must be.
They are positive at the start, negative in the middle and positive again at the end: a bowl, or U shape, the same on both sides.
A straight line is the wrong shape: a curve that bends upwards would suit the data better.

Your go. The first step is given. The residuals of a regression line at x = 1 to 7 are −3, +2, −1, +3, +2, −1 and −2. Do they suggest a straight line is appropriate?

First check they add to 0.
Look at the signs and sizes in order of x. Is there a shape?

Check the answer

The total is −3 + 2 − 1 + 3 + 2 − 1 − 2 = 0. The signs go −, +, −, +, +, −, − with no long run of one sign and no growth in size: no pattern, so a straight line is a fair model for this range. A common slip is to look only at the sizes and declare the line “bad” because some residuals are as large as 3: what matters is the shape, and whether 3 is large compared with the values of y.

With a GDC (optional)

After you run the linear regression command, many GDCs store the residuals in a list (often named RESID). Plot that list against your x list as a scatter graph: if the residuals form a clear arch or a bowl, a straight line is the wrong shape; if they fan out wider as x rises, predictions are less reliable where the fan is wide. If they look like a shapeless cloud around 0, a straight line is a good model.

The calculator will give a prediction at any x without comment, so the decision to trust it is yours: check whether x is inside the range of your data. Everything in this lesson can be done without a GDC.

Check before you go on

1. The residuals of a regression line at x = 1 to 7, in order, are 0, +1, −1, +2, −2, −3 and +3. Which is the best reading?

There is no pattern: the signs keep changing, so the line is equally trustworthy for every x in the data. A prediction at x = 7 deserves less trust than one at x = 2. The residuals form a U, so a curve that bends upwards would fit better than the straight line. The line is a perfect fit, because the residuals add to 0 and so cancel each other out exactly.

Your turn

Start at Level 1 and go as far as you can. Type negative numbers with a minus sign, for example -7. A wrong answer gets a reason, not just a cross: read it, then try again.

Level 1 · Getting started

2. A line is y = 2.5x + 10. A dot is at (8, 34). What is its residual (dot minus line)?

3. A regression line has five residuals. Four of them are 1.5, −2, 0.5 and 3. What is the fifth?

Level 2 · Building

4. A line is y = 3x + 4. Two dots are (6, 25) and (9, 29). Find the total of the squares of their two residuals.

5. For the heights of a young tree measured each month, r = 0.97, but the residuals of the regression line form a clear U shape when plotted against x. Which conclusion is best?

The linear model is appropriate, because r is so close to 1 that the line must fit the data well. r must have been worked out wrongly, because a U shape in the residuals can only come with a low r. The U shape is only chance, because r is so close to 1 that no pattern in the residuals can be real. The line follows the trend but misses a bend: a curve would model the data better.

Level 3 · Stretch

6. A regression line fitted to data with x from 5 to 20 predicts y = 45 at x = 40. A later measurement at x = 40 is 36. By what percentage of the measured value was the prediction too high?

7. Two analysts fit the same data. A uses a straight line, with r = 0.91 and residuals that form an arch. B uses a curve whose residuals are shapeless. A manager wants a prediction for an x inside the data. Which is the best advice?

Use A's line, because a straight line is simpler to explain and r = 0.91 is high. Use B's curve, because its residuals show it follows the data with no systematic miss. Use A's line, because the residuals of any regression line add to 0, so its errors cancel over the data. Use either: both were fitted to the same data, so their predictions inside the data must agree.

Something to think about

A company fits a regression line to its first four months of sales and finds r = 0.99. The manager says: “r is nearly perfect, so we can forecast next year’s sales from the line.” Give two different reasons the forecast may still be poor.

One way to think about it

First, next year is far outside the four months of data, so the pattern may bend, level off or have a season in it that four months cannot show. Second, only four dots were used, so a high r can come from very little data, and a residual plot of four points can hide a curve. A high r says the line fits these dots, not that it will go on fitting.

Exit ticket

8. A line is fitted to data with x from 1 to 7. r = 0.95 and the residuals, in order of x, are −3, 0, +2, +2, +2, 0 and −3. A student wants to use the line to predict y at x = 12. Which advice is best?

Use it: r = 0.95 is close to 1, so the line fits the data well everywhere, inside and outside the data. Use it: the residuals add to 0, so the errors above and below the line cancel out in the end. Do not use it, but only because x = 12 is outside the data; inside the data this line is a sound model. Do not use it: the residuals form an arch, and x = 12 is outside the data.