At a glance. One lesson of about 60 minutes. It assumes the sampling methods of Lessons 2 and 3 and the idea of bias from Year 9. Students name sources of bias, work out a response rate and the range that non-responders allow, weight a biased estimate, and use the wobble rule to size a sample.
Calculator: scientific calculator. The rule of thumb for sample size needs a square root, so a scientific calculator is useful for the root. Perfect squares are used so the root can be found by hand. The rule 100 ÷ √n is the usual 95% margin for a percentage near 50%.
What students will learn
Know
- Sampling bias, self-selection, non-response bias and wording bias.
- Response rate is replies divided by people asked.
- The rough margin of error is about 100 divided by the square root of n points.
Understand
- A larger sample shrinks wobble and not bias.
- Non-response can move a figure a long way when the response rate is low.
- Who is asked matters more than how many.
Do
- Say which kind of bias a design has and why.
- Find the range the non-responders allow.
- Weight a figure by the shares of a population.
MYP assessment criteria
Wording paraphrased. What this lesson gives you evidence for, and what to look for:
- Criterion A. Knowing and understanding: types of bias, response rate and the wobble rule (questions 1 to 8).
- Criterion B. Investigating patterns: the second figure. Ask for what happens to each band before changing the sample size.
- Criterion C. Communicating: explaining a bias in a full sentence with the group it affects (the worked examples).
- Criterion D. Applying mathematics in real-life contexts: phone-ins, questionnaires, polls and product surveys.
The one thing to do with the figure
Name the lean before reading the number. In the first figure, ask which group goes quiet as the slider moves left, and which way the estimate should move. Take a prediction, then slide.
In the second figure, cover the fair band and ask whether collecting ten times as many responses fixes the phone-in. Uncover it and ask what the fair method changed.
A possible order
| 5 min | Warm-up from the student page, in silence, then answers. |
|---|---|
| 7 min | Hook: the phone-in. Collect predictions about ten times as many callers. |
| 10 min | First figure and key idea: kinds of bias and the response rate. |
| 10 min | Second figure, the warning card and the worked examples; pairs finish the faded example. |
| 5 min | The check. If many choose B, ask what a website poll with a million clicks would still lack. |
| 18 min | Your turn, Levels 1 to 3, at each student's own pace. |
| 5 min | Something to think about, then the exit ticket, question 8. |
The answers
| Warm-up | 12; 9; 30%. |
|---|---|
| Hook | No: callers choose themselves, so ten times as many callers is ten times as many self-selected people. The result would have to come from a sample chosen so that everyone, or a fair share of every group, could be asked. |
| Worked examples | The gym sample leans towards evening users; the 500 forms allow 18% to 88%. |
| Faded example | √625 = 25 and 100 ÷ 25 = 4 points. |
| 1. Naming a bias | A. B blames size; C names the wrong method; D says readers are outside the population. |
| 2. A fair question | C. A, B and D each use words that lead. |
| 3. Response rate | 40. 180 ÷ 450. |
| 4. What non-responders could change | 75. (150 + 150) ÷ 400. |
| 5. Estimating when a group is missed | 51. 0.7 × 60 + 0.3 × 30. |
| 6. Choosing a better design | B. A only enlarges the same biased sample; C asks the wrong people; D changes the look, not the method. |
| 7. Sample size from a wobble | 2500. n = (100 ÷ 2) squared. |
| 8. Naming the bias in a new case | D. A names wording; B names non-response; C names the wrong method. |
What each wrong answer tells you
| They give | What it usually means | What to do |
|---|---|---|
| B, C or D in question 1 | Blames size, names the wrong method, or says the audience is outside the population. | Ask: who is likely to click on a website poll, and who is not? |
| A, B or D in question 2 | Picks wording that leads while sounding polite. | Ask: which word tells you which answer the writer wants? |
| 60, 2.5 or 270 in question 3 | Gives the non-response rate, inverts the fraction, or gives a count. | Write replies over asked and say which is on top. |
| 60, 37.5 or 62.5 in question 4 | Gives the share of replies, the lowest case, or the response rate. | Ask: how many people are silent, and what is the most they could add? |
| 60, 45 or 90 in question 5 | Gives only the landline figure, takes the plain average, or adds the percentages. | Use a bar split 70 : 30 and ask which part is bigger. |
| A, C or D in question 6 | Chooses more of the same, asks the wrong group, or changes only the look. | Ask: does this change who is in the sample? |
| 50, 200 or 4 in question 7 | Stops before squaring, multiplies, or squares the wobble. | Say the rule as a sentence: 100 divided by the root of n. |
| A, B or C in question 8 | Blames wording, non-response, or says it is stratified. | Ask: who in the school could not have been given the form? |
Questions worth asking
- Why can a survey of a million people be badly biased? A huge sample of a biased method is still biased.
- What is the difference between a bias and a wobble? A wobble goes both ways and shrinks with n; a bias leans one way and does not.
- If only a third of forms come back, what should you do? Follow up non-responders and be cautious about the result.
- How could a question be worded fairly? Name the options evenly without hints about which is better.
Differentiation
A way in for anyone stuck
- Give a three-column list: who could be asked, who replied, how was it worded.
- Provide a table of 100 ÷ √n for n = 25, 100, 400 and 900 to read from.
- Use a number line from 0 to 100 and let students mark the range that non-responders allow.
Stretch
- A survey has a response rate of 80%. How far could the true percentage be from the percentage among replies? Give the largest possible gap.
- To halve the wobble, by what factor must n grow? Show using the rule.
Students learning in English as an additional language
“Bias” here means a lean in one direction. “Self-selection” means people choose themselves. “Response rate” is the share who answered. “Loaded” means a word that carries a hidden opinion.
| Word | Plain meaning | Example |
|---|---|---|
| bias | a lean that pushes results one way every time | a clock that always runs five minutes fast |
| self-selection | taking part by your own choice, so the sample is made of volunteers | people who choose to post a review of a restaurant |
| non-response | members chosen for a sample who do not reply | questionnaires never returned |
| response rate | the share of people asked who actually reply | replies divided by forms sent |
| leading question | a question worded to push towards one answer | a question that hints at the answer wanted |
| random wobble | the chance difference between a sample result and the truth, which shrinks as n grows | the gap between two fair samples |
Sentence starters for explaining aloud or in writing:
- This could be biased because … could not be asked.
- The response rate is …, so the people who did not reply could change the result by ….
- A larger sample would not help because the problem is ….
Two things not to say
“A big sample means the survey is right.” A big biased sample is wrong with more confidence.
“Biased means somebody cheated.” A design can lean without anyone intending it.
Questions to set
Extra practice in three tiers, including one question from earlier work for spacing. Every answer has been checked.
1Fluency
The idea on its own, with friendly numbers. Set these first.
- Response rate300 forms sent, 90 returned. Response rate?
90 ÷ 300 = 30%. - Wobblen = 256. Wobble in points?
100 ÷ 16 = 6.25.
2In context
The same idea inside a situation, where the first job is deciding what to calculate.
- ContextA survey asks 200 people and 120 reply; 72 of those say yes. What is the lowest percentage of all 200 who could say yes?
72 ÷ 200 = 36%. - Earlier work (Lesson 3)A stratified sample of 60 is taken from groups of 120, 180 and 300. How many from the 180 group?
180 ÷ 600 × 60 = 18.
3Challenge
Reasoning, generalizing and working backwards.
- ReasoningA survey has a wobble of 4 points and a bias of 15 points. A second has a wobble of 10 points and no bias. Which is closer to the truth, usually?
The second, usually: its error is within about 10 points, while the first is about 15 points off, give or take 4. - Working backwardsA sample size gives a wobble of 5 points. What size gives a wobble of 1 point?
A wobble of 5 means the root of n is 20 (n = 400). A wobble of 1 needs a root of 100, so n = 10000.
Practicalities
Works on a phone or a shared screen. Nothing a student types is saved or sent anywhere.
Both figures work with the keyboard (tab to a slider, then the arrow keys) and by touch. Under each one, “Show me the steps” gives the same states in words.
No device? Have pairs write three versions of one survey question, one fair and two leading, and swap them.