Year 11 Extended · Unit 5 Sampling, correlation and regression · Lesson 4 of 5

The regression line: the line that makes the squared gaps smallest

Many lines pass through the mean point. The regression line of y on x is the one for which the total of the squared vertical gaps is smallest, and its gradient and intercept come from the same sums as r. This lesson finds it by formula once, and then with a GDC.

Statistics and ProbabilityGDC optional

Warm-up

Three quick ones from earlier work. Try each in your head or on paper before you open the answer.

  1. A line is y = 3x + 8. Find y when x = 5.
    Answer23, because 3 × 5 + 8 = 23.
  2. The sum of dx² is 20 and the sum of dx × dy is 30. What is the sum of dx × dy divided by the sum of dx²?
    Answer30 ÷ 20 = 1.5.
  3. From Lesson 3: r = 0.6, the sum of dx² is 16 and the sum of dy² is 25. What is the sum of dx × dy?
    Answer0.6 × √(16 × 25) = 0.6 × 20 = 12.
Words used in this lesson
regression line
the line of y on x that makes the total of the squared vertical gaps smallest. For example: the line used to predict a plant's height from its hours of light.
gradient
how much y changes for a rise of 1 in x. For example: a gradient of 1.5 means y rises by 1.5 for each extra 1 in x.
intercept
the value of y where the line crosses the y-axis. For example: the height at 0 hours of light.
residual
the gap from a dot to the line: the dot's y minus the line's y. For example: a plant 2 cm below the line.
least squares
the method that chooses the line with the smallest total of squared gaps. For example: the rule that picks the regression line.

Which line is the best line?

Five plants of the same kind were grown with x hours of daily light, from 1 to 5, and their heights y were recorded in centimetres: 14, 17, 17, 22 and 25. The mean point is (3, 19). A line through the mean point can be tilted like a seesaw about that point. The gap from a dot to a line is how far the dot is above or below it, measured straight up or down.

Predict first: as you tilt the line, the total of the squares of the five gaps changes. Between gradient 1 and gradient 4, at roughly which gradient do you think it is smallest? Then drag the slider and watch the small graph.

2
2gradient
13y-intercept of the line
10total of the squared gaps

Gradient 2: the line through the mean point is y = 2x + 13, and the squares of the five gaps add to 10.

Show me the steps
  1. Gradient 2: the line is y = 2x + 13. The gaps are −1, 0, −2, 1 and 2 (dot minus line), and their squares add to 1 + 0 + 4 + 1 + 4 = 10.
  2. Gradient 3: the line is y = 3x + 10 and the squares of the gaps add to 6, smaller than 10.
  3. Gradient 2.7: the line is y = 2.7x + 10.9 and the squares add to 5.1, the smallest of all the gradients. This is the regression line.
  4. Gradient 4: the line is y = 4x + 7 and the squares add to 22. Tilting too far makes the total grow again, so there is one lowest point.

The regression line of y on x is the line that makes the total of the squared gaps smallest.

  • Its gradient is b = (sum of dx × dy) ÷ (sum of dx²), the same sums as in r. Equivalently b = r × (spread of y) ÷ (spread of x).
  • It passes through the mean point, so its intercept is a = (mean of y) − b × (mean of x). The line is y = a + bx.
  • Because it passes through the mean point, the gaps (the residuals) add to 0.
  • It is for predicting y from x, inside the data. A GDC gives it from two lists.

Many lines go through the mean point. Which one is the regression line, and what decides it? Think about the gaps, and what happens to them once they are squared. And a regression line can be worked out for any dots, however poorly they follow a line, so a regression line is not evidence that a straight line is a good model. That is the work of the next lesson.

Worked examples

Dots (2, 6), (3, 7), (5, 11), (6, 11) and (9, 15). Find the regression line of y on x.

  1. The means are 5 and 10. dx = −3, −2, 0, 1, 4 and dy = −4, −3, 1, 1, 5.
  2. Products: 12, 6, 0, 1, 20 which add to 39. Squares of dx: 9, 4, 0, 1, 16 which add to 30.
  3. b = 39 ÷ 30 = 1.3 and a = 10 − 1.3 × 5 = 3.5. The line is y = 1.3x + 3.5.

For a data set the mean of x is 14, the mean of y is 62, the sum of dx × dy is −96 and the sum of dx² is 48. Find the regression line.

  1. b = −96 ÷ 48 = −2.
  2. a = 62 − (−2) × 14 = 62 + 28 = 90.
  3. The line is y = 90 − 2x. The gradient is negative because the sum of products is negative.

Your go. The first step is given. Dots (1, 9), (2, 10), (4, 14), (5, 15) and (8, 17). Find the regression line of y on x.

  1. The mean of x is 20 ÷ 5 = 4.
  2. Find the mean of y, then dx, dy, the sums, b and a.
Check the answer

The mean of y is 65 ÷ 5 = 13. dx = −3, −2, 0, 1, 4 and dy = −4, −3, 1, 2, 4. Products 12, 6, 0, 2, 16 total 36 and the squares of dx total 30, so b = 36 ÷ 30 = 1.2 and a = 13 − 1.2 × 4 = 8.2. The line is y = 8.2 + 1.2x. A common slip is to divide by the sum of dy² instead of dx²: that is the gradient of the other line.

With a GDC (optional)

Enter the x values in one list and the y values in another, then use the linear regression command (often under the stats or calculation menu, named something like “LinReg”, with the form y = ax + b). Careful with the letters: in that form a is the gradient and b the intercept, the other way round from y = a + bx in this lesson. The calculator usually gives r as well.

Always check two things by hand: the line should pass through the mean point of your data, and its gradient should have the same sign as r. If you type the lists in the wrong order, the calculator gives the line of x on y, which is a different line. Everything in this lesson can be done without a GDC.

Check before you go on

1. A line through the mean point of five dots has gaps (dot minus line) of 3, −1, −4, 0 and 2. What is the total of the squared gaps?

Your turn

Start at Level 1 and go as far as you can. Type negative numbers with a minus sign, for example -7. A wrong answer gets a reason, not just a cross: read it, then try again.

Level 1 · Getting started

2. For a data set the sum of dx × dy is 90 and the sum of dx² is 36. Find the gradient of the regression line of y on x. Give it as a decimal.

3. A regression line has gradient 1.5 and the mean point is (8, 47). Find its y-intercept.

Level 2 · Building

4. Dots: (10, 25), (20, 34), (30, 37), (40, 44) and (50, 50). Find the y-intercept of the regression line of y on x.

5. A GDC gives the regression line y = 1.6x − 12 for data with x from 10 to 40. Predict y when x = 25.

Level 3 · Stretch

6. For a data set r = 0.6, the spread (standard deviation) of x is 5 and of y is 10. The means are 20 and 55. Using b = r × (spread of y) ÷ (spread of x), predict y when x = 25.

7. Why does the regression line square the gaps before adding them, instead of adding the gaps as they are?

Something to think about

Two students fit a line to the same data. One uses the formulas and the other uses a GDC and types the lists the wrong way round, so the calculator regresses x on y. Both lines pass through the mean point. Will the two lines be the same line? How could you tell which one predicts y from x?

One way to think about it

They are different lines, unless the dots lie exactly on a line. Both pass through the mean point, but the regression of x on y makes the horizontal gaps smallest, not the vertical ones, so its gradient (as drawn with y up) is steeper. To predict y from x you need the line that minimizes the vertical gaps: the one with y as the output, which is the one with gradient equal to the sum of products divided by the sum of dx².

Exit ticket

8. Five dots have x = 1, 2, 3, 4 and 5. Lines P and Q both pass through their mean point. The gaps (dot minus line) are 0, −2, −2, 0 and 4 for P, and 2, −1, −2, −1 and 2 for Q. Which statement is right?