Jugami › Statistics and Probability › Random Variables and Sampling
Statistics · Unit 4
Random Variables and Sampling
One game, one survey or one sample is a gamble. Thousands of them follow rules you can predict, and those rules tell you what a sample can (and can't) say about everyone.
In this unit you'll learn what a random variable is, how to find its expected value, variance and standard deviation, when to use the binomial formula, how good samples are chosen and how bad ones go wrong, and how the central limit theorem explains why averages of random samples follow the normal curve.
1936: 2.4 million replies, wrong winner
In the 1936 US presidential election, the magazine Literary Digest mailed mock ballots to about 10 million people, using lists of its own readers, car owners and telephone users. About 2.4 million sent them back, one of the largest polls ever taken. It predicted that Alf Landon would beat President Franklin Roosevelt with about 57% of the vote.
Roosevelt won with about 61% and carried every state except Maine and Vermont. Meanwhile a young pollster named George Gallup, using a sample of about 50,000, correctly predicted a Roosevelt win.
How can 2.4 million answers lose to 50,000? Because how a sample is chosen matters far more than how big it is. This unit shows why, first by treating chance outcomes as numbers (random variables), then by watching what happens when you take many random samples.
Random variables and their distributions
A random variable is a number whose value depends on chance: the number of heads in 3 flips, the number on a die, how much you win in a game. We name random variables with capital letters like X, and their possible values with small letters like x.
A discrete random variable has values you can list (0, 1, 2, …). Its probability distribution gives every possible value with its probability. Two rules always hold:
- Every probability is between 0 and 1.
- The probabilities add up to exactly 1.
(A continuous random variable, like a height or a time, can take any value in a range. Its probabilities are areas under a curve, like the normal curve from Unit 3.)
| x | 0 | 1 | 2 | 3 |
|---|---|---|---|---|
| P(X = x) | 0.10 | 0.35 | 0.40 | 0.15 |
From the table, P(X ≥ 2) = 0.40 + 0.15 = 0.55: 55% of households have two or more cars.
Expected value
The expected value (or mean) of X is the long-run average value you'd get if you repeated the chance process over and over. Multiply each value by its probability and add:
For the cars: E[X] = 0(0.10) + 1(0.35) + 2(0.40) + 3(0.15) = 0 + 0.35 + 0.80 + 0.45 = 1.6 cars. No household owns 1.6 cars. The expected value is an average, so it doesn't have to be a possible value.
Worked example: is this game fair?
You pay $1 to roll a die. If you roll a six, you get $6. Let X be your net gain.
- A six: you get $6 but paid $1, so X = +$5, with probability 1/6.
- Anything else: you lose your $1, so X = −$1, with probability 5/6.
- E[X] = 5 · (1/6) + (−1) · (5/6) = 5/6 − 5/6 = $0.
An expected value of 0 means the game is fair: over many games neither you nor the person running it comes out ahead. A negative expected value favors the house.
Worked example: a school raffle
A club sells 500 raffle tickets at $2 each (made-up numbers). There is one $300 prize and two $50 prizes. You buy one ticket.
- Net gains: win $300 → +$298 (probability 1/500); win $50 → +$48 (probability 2/500); win nothing → −$2 (probability 497/500).
- Expected prize money: 300(1/500) + 50(2/500) = 0.60 + 0.20 = $0.80.
- E[net gain] = $0.80 − $2 = −$1.20 per ticket.
The club expects to raise 500 × $1.20 = $600, which is the $1,000 in ticket sales minus $400 in prizes. That's fine for a fundraiser: you're paying $1.20 on average to support the club.
The expected value is a long-run idea. It connects straight to the law of large numbers: play enough games and your average gain per game gets close to E[X]. Try it.
With the raffle, the line jumps wildly every time someone wins a prize, and it takes many thousands of tickets to settle near −$1.20. Rare, big payouts make a game's results spread out, which is what the next section measures.
Variance and standard deviation of a random variable
Two games can have the same expected value and very different risk. The variance of X measures the spread: the expected squared distance from the mean. The standard deviation is its square root, in the original units.
This is the same idea as the SD of a data set in Unit 3, but each squared deviation is weighted by its probability instead of being averaged over n − 1.
| x | P(x) | x · P(x) | (x − 1.6)² | (x − 1.6)² · P(x) |
|---|---|---|---|---|
| 0 | 0.10 | 0 | 2.56 | 0.256 |
| 1 | 0.35 | 0.35 | 0.36 | 0.126 |
| 2 | 0.40 | 0.80 | 0.16 | 0.064 |
| 3 | 0.15 | 0.45 | 1.96 | 0.294 |
| Total | 1 | μ = 1.6 | Var = 0.74 |
So σ = √0.74 ≈ 0.86 cars.
Worked example: same mean, different risk
- Fair die game: X = +5 or −1, μ = 0. Var = (5 − 0)²(1/6) + (−1 − 0)²(5/6) = 25/6 + 5/6 = 5, so σ = √5 ≈ $2.24.
- Raffle: μ = −1.20. Var = (298 + 1.2)²(1/500) + (48 + 1.2)²(2/500) + (−2 + 1.2)²(497/500) ≈ 179.04 + 9.68 + 0.64 = 189.36, so σ ≈ $13.76.
The raffle's SD is huge compared with the $2 ticket because almost everyone loses $2 and a few people win a lot. A large SD means any one result is hard to predict.
The binomial distribution
Many random variables count successes: right answers when guessing, free throws made, heads in a row of flips. When the setting passes four checks, called BINS, the count X has a binomial distribution:
- Binary: each trial is a success or a failure.
- Independent: one trial's result doesn't change another's chances.
- Number: the number of trials n is fixed in advance.
- Same chance: the probability of success p is the same on every trial.
To get exactly k successes in n trials, choose which k trials succeed (that's nCk, the combinations from Unit 2), then multiply the chances:
Worked example: guessing on a quiz
A 10-question quiz has four choices per question. You guess on every question, so n = 10 and p = 1/4 = 0.25. Let X be the number right.
- BINS: right or wrong (B), guesses don't affect each other (I), n = 10 (N), p = 0.25 every time (S). Binomial.
- P(X = 3) = 10C3 · 0.25³ · 0.75⁷ = 120 × 0.015625 × 0.1335 ≈ 0.250.
- P(X = 0) = 0.75¹⁰ ≈ 0.056. P(X ≤ 2) = P(0) + P(1) + P(2) ≈ 0.056 + 0.188 + 0.282 = 0.526.
- Passing needs 6 or more: P(X ≥ 6) ≈ 0.020, about 1 chance in 50.
- Mean np = 10 × 0.25 = 2.5 right; SD = √(10 × 0.25 × 0.75) = √1.875 ≈ 1.37.
Worked example: free throws
A player makes 70% of free throws and takes 5 (assume the shots are independent). X = shots made, n = 5, p = 0.7.
- P(X = 4) = 5C4 · 0.7⁴ · 0.3 = 5 × 0.2401 × 0.3 ≈ 0.360.
- P(X = 5) = 0.7⁵ ≈ 0.168, so P(at least 4) ≈ 0.360 + 0.168 = 0.528.
- Mean = 5 × 0.7 = 3.5 shots; SD = √(5 × 0.7 × 0.3) = √1.05 ≈ 1.02.
Try 20 coin flips with the normal curve on. The mean is 10 and the SD is √5 ≈ 2.24, and the bell curve with that mean and SD hugs the bars. A common rule of thumb: the normal curve is a good fit when np ≥ 10 and n(1 − p) ≥ 10. With the quiz (np = 2.5) the bars are skewed and the curve fits badly.
A geometric nod. If you keep going until the first success, there is no fixed n, so the count of trials isn't binomial. It is geometric: P(first success on trial k) = (1 − p)k − 1 · p. The chance the first six comes on the 3rd roll is (5/6)² · (1/6) = 25/216 ≈ 0.116, and on average you wait 1/p = 6 rolls.
Populations, samples and bias
The population is the whole group you want to know about. A sample is the part you actually measure. A number that describes the population is a parameter (usually unknown); a number computed from a sample is a statistic, which we use to estimate the parameter.
| Population (parameter) | Sample (statistic) | |
|---|---|---|
| Mean | μ | x̄ |
| Standard deviation | σ | s |
| Proportion | p | p̂ (“p-hat”) |
Statisticians use chance to choose samples, because chance plays no favorites:
- Simple random sample (SRS): every group of n individuals is equally likely to be chosen, like drawing names from a hat.
- Stratified sample: split the population into groups that are alike (strata), such as grade levels, and take an SRS from each.
- Cluster sample: split the population into groups that are like small copies of it (clusters), such as homerooms, pick some clusters at random and survey everyone in them.
- Systematic sample: pick a random start, then every kth person on a list.
A sampling method is biased if it tends to give results that miss the truth in the same direction. Three common causes:
- Convenience sampling: asking whoever is easy to reach, like the people near you in the cafeteria.
- Voluntary response: people choose themselves, as in call-in or online polls. People with strong feelings, often negative, are the ones who answer.
- Undercoverage: some part of the population can't be chosen at all, such as people left off the list you sample from.
Even a good sample can go wrong after it's chosen: nonresponse (chosen people don't answer) and response bias (leading questions, or people not telling the truth).
Random samples bounce around, but they bounce around the truth. The biased methods miss in the same direction every time, and taking more or bigger samples doesn't help. That's what sank the Literary Digest: its lists of car owners, telephone users and its own readers leaned toward better-off voters during the Depression, and a later study found that who chose to mail back a ballot mattered even more.
Sampling distributions
A statistic changes from sample to sample. That's sampling variability, and it isn't a mistake. The sampling distribution of a statistic is the pattern of its values over all possible random samples of the same size. For the sample mean x̄ from random samples of size n:
- Center: the mean of all the x̄ values is μ, the population mean. x̄ is unbiased.
- Spread: the SD of the x̄ values, called the standard error, is σ/√n. Bigger samples give less variable means.
The square root matters. To cut the standard error in half you need four times the sample size, not twice. (For a sample proportion p̂ the same idea gives center p and SD √(p(1 − p)/n).)
Bias and variability are different problems. Variability is about how much the darts scatter; bias is about whether they're centered on the bullseye. A bigger random sample shrinks the scatter; nothing about size fixes a biased method.
The central limit theorem
Here is the surprise. Whatever shape the population has (skewed, flat, two peaks), the distribution of x̄ for large random samples is approximately normal. This is the central limit theorem (CLT):
For random samples of size n from any population with mean μ and SD σ, when n is large the sampling distribution of x̄ is approximately normal, with mean μ and standard error σ/√n.
Conditions to check:
- Random: the data come from a random sample (or a randomized experiment).
- Independent: when sampling without replacement, the sample is no more than 10% of the population.
- Large enough: if the population is normal, any n works. Otherwise a common rule of thumb is n ≥ 30, and more if the population is very skewed or has outliers.
The CLT is why the normal curve shows up all over statistics: averages of many random pieces behave normally even when the pieces don't. Once you know x̄ is approximately normal, you can use z-scores and normal areas from Unit 3. Just use the standard error as the SD.
Worked example: average wait time
Wait times at a coffee shop are skewed right, with mean μ = 4 minutes and SD σ = 3 minutes (made-up numbers). You take a random sample of 36 customers. How likely is a sample mean above 5 minutes?
- Conditions: a random sample, 36 is far less than 10% of all customers, and n = 36 ≥ 30, so the CLT applies even though the wait times are skewed.
- x̄ is approximately normal with mean 4 and SE = 3/√36 = 3/6 = 0.5 minutes.
- z = (5 − 4) ÷ 0.5 = 2.
- By the 68–95–99.7 rule about 2.5% of the area is above z = 2 (the exact normal area is 2.28%). So P(x̄ > 5) ≈ 0.023.
A single customer waiting more than 5 minutes is common. An average of 36 customers that high would be rare.
Common mistakes
Expecting the expected value. E[X] is a long-run average. It may be impossible in one trial (1.6 cars), and a single game can turn out far from it.
Forgetting the cost. In game problems, decide whether X is the payout or the net gain. Net gain = payout − cost.
Using the binomial formula without BINS. “Until the first success” has no fixed n; drawing without replacement from a small group breaks independence.
Leaving out nCk. pk(1 − p)n − k is the chance of one particular order. Multiply by nCk for all the orders.
Giving the variance as the SD. np(1 − p) is the variance. Take the square root.
Thinking a bigger sample fixes bias. 2.4 million biased replies lost to 50,000 well-chosen ones. Size reduces variability, not bias.
Saying the population becomes normal. The CLT is about the distribution of sample means. The population, and any one sample, keep their own shape.
Dividing by n instead of √n. The standard error is σ/√n.
Practice
People behind the ideas
1657 Christiaan Huygens publishes De ratiociniis in ludo aleae (“On reasoning in games of chance”), working out the fair value of a gamble: the idea we now call expected value.
1713 Jacob Bernoulli's Ars Conjectandi (“The Art of Conjecturing”) is published in Basel, eight years after his death, by his nephew Nicolaus Bernoulli. It develops Huygens' expected value, studies repeated success-or-failure trials (now called Bernoulli trials) and proves the first version of the law of large numbers, which Bernoulli called his “golden theorem” and said took him twenty years.
1733 Abraham de Moivre uses the curve we now call the normal curve to approximate the number of heads in many tosses of a fair coin: the first glimpse of the central limit theorem.
1810 Pierre-Simon Laplace proves the first general central limit theorem, in papers of 1810 and 1811. His 1812 book Théorie analytique des probabilités extends de Moivre's result, approximating the binomial distribution by the normal.
1901 Aleksandr Lyapunov proves the theorem in general terms. The name came later: George Pólya called it the “central” limit theorem in 1920, because of its central place in probability.
1936 The Literary Digest poll gets about 2.38 million replies from some 10 million ballots and predicts a Landon win with about 57%. Roosevelt wins with about 61% and 523 electoral votes. George Gallup's American Institute of Public Opinion, with a sample of about 50,000, correctly picks Roosevelt. A 1988 study by Peverill Squire concluded that nonresponse (who chose to reply) was the main cause of the Digest's error, together with its unrepresentative lists. The magazine went out of business in 1938.
Quiz
Ten questions. Pick an answer for each, then check.
Talk about it
- A state lottery has a negative expected value for every ticket, yet millions of people play. Give some reasons someone might reasonably buy a ticket anyway. When does expected value matter most?
- An insurance company and its customers can both be “winning”. How can that be, when the company's expected profit on each policy is positive?
- A news website runs a poll: “Should the city build a new stadium? Click to vote.” 40,000 people vote. Would you trust the result? What would you do instead to find out what city residents think?
- Why might a survey that only reaches people by phone miss some groups today? Who would be left out, and how could that change the answer?
- The CLT says averages are predictable even when individuals aren't. Where else might this matter: in sports, weather, medicine or business?
Sources
- Wikipedia, Ars Conjectandi: published in 1713 in Basel, eight years after Jacob Bernoulli's death, by his nephew Nicolaus Bernoulli; the first version of the law of large numbers, which Bernoulli called his “golden theorem” and said took twenty years; Bernoulli trials; and its development of Huygens' expected value from De ratiociniis in ludo aleae (1657).
- Wikipedia, Central limit theorem: history, quoting Henk Tijms, Understanding Probability: de Moivre's 1733 article approximating the number of heads with the normal curve; Laplace's Théorie analytique des probabilités (1812) approximating the binomial by the normal; Lyapunov's general proof (1901); and Pólya's 1920 name “central limit theorem”.
- Wikipedia, Pierre-Simon Laplace: the first general central limit theorem, proved in papers of 1810 and 1811.
- Wikipedia, The Literary Digest, citing David Freedman, Robert Pisani and Roger Purves, Statistics (4th ed.), pp. 335–336, and Peverill Squire, “Why the 1936 Literary Digest Poll Failed”, Public Opinion Quarterly 52 (1), 1988: about 10 million ballots and 2.38 million replies; lists of readers, car registrations and telephone users; the 57% Landon prediction; Roosevelt's 60.8% and 523 electoral votes, winning every state but Maine and Vermont; Gallup's sample of about 50,000; nonresponse bias as the main cause; the magazine folding in 1938.
- Wikipedia, George Gallup: his American Institute of Public Opinion correctly predicted Roosevelt's 1936 win.
- All binomial, geometric and normal probabilities on this page were computed exactly and rounded; the coffee-shop answer (2.28%) is the standard normal area above z = 2.
- All data sets on this page (household cars, the raffle, the spinner, the 400 students, the three populations in the CLT sampler and the coffee-shop wait times) are made up for teaching.