Jugami › Statistics and Probability › Data Displays and Distributions
Statistics · Unit 3
Data Displays and Distributions
A list of numbers hides its story. Draw it, and you can see where the values pile up, how far they spread and whether anything is out of place.
In this unit you'll learn when to use a dot plot, a stem-and-leaf plot, a histogram or a box plot, how to describe a shape, how to measure spread with the range, IQR, standard deviation and variance, how graphs can mislead, and how the normal curve, the 68–95–99.7 rule and z-scores work.
1835: Quetelet's “average man”
In 1835 the Belgian astronomer and statistician Adolphe Quetelet published a book called Sur l'homme (“On man”). He had collected measurements of people, such as heights and weights, and noticed that they piled up around a middle value and thinned out on both sides in a bell shape. Astronomers already knew that curve: it described the small errors in repeated measurements. Quetelet was the first to use it for something other than errors. He imagined an “average man” at the center, with real people spread around him.
The idea came with a warning that still matters. A middle value tells you nothing about the spread, and a bell shape is something you have to check, not assume. This unit gives you the tools to do both: graphs to see the shape, and numbers to measure the spread.
Choosing a display
All the displays in this unit are for quantitative data: numbers you can measure, like times, heights or scores. (Categories such as favorite sport go in a bar chart or a two-way table.) Each one trades detail for a clearer overall picture.
Stem-and-leaf plots
A stem-and-leaf plot is a dot plot made of digits. Split each value into a stem (here, the tens digit) and a leaf (the ones digit), then write the leaves in order beside their stem. These 15 made-up quiz scores are 52, 61, 64, 67, 70, 72, 72, 75, 78, 81, 83, 85, 88, 90, 97:
| Stem | Leaves |
|---|---|
| 5 | 2 |
| 6 | 1 4 7 |
| 7 | 0 2 2 5 8 |
| 8 | 1 3 5 8 |
| 9 | 0 7 |
Turn your head sideways and the rows of leaves form a histogram, yet every original value is still there. You can read off the median straight away: with 15 values it is the 8th, which is 75. Always include a key, because “7 | 2” could mean 72, 7.2 or 720.
| Display | Keeps every value? | Good for | Watch out for |
|---|---|---|---|
| Dot plot | Yes | Small data sets; seeing gaps, clusters and repeats | Gets crowded past about 50 values |
| Stem-and-leaf | Yes | Small data sets done by hand; finding the median | Needs a key; awkward with decimals or wide ranges |
| Histogram | No | Large data sets; the overall shape | The picture changes with the bin width |
| Box plot | No | Comparing groups; spotting outliers | Hides shape: two peaks can look like one box |
Histograms and bin width
A histogram looks like a bar chart, but it isn't one. In a bar chart each bar is a category, the bars have gaps and you could put them in any order. In a histogram each bar is a range of numbers called a bin, the bins are in number order, and the bars touch because the ranges join up. All bins have the same width. Here a value on an edge goes in the bin to its right, so 10 counts in the 10–15 bin, not 5–10.
The choice of bin width changes the picture. Too narrow and the histogram is a spiky mess where every bump looks important. Too wide and real features get blurred together. There's no single right width, so try a few.
With 5-minute bins the commute times have two peaks, around 10 minutes and around 28 minutes. Perhaps some students walk and others ride a bus. With 10-minute bins the two groups merge into a single lump and that story vanishes.
Box plots, fences and outliers
A box plot draws the five-number summary: minimum, Q1, median, Q3, maximum. The box runs from Q1 to Q3, so it holds the middle half of the data, and a line marks the median. We find quartiles the same way as in Unit 1: Q1 is the median of the lower half and Q3 the median of the upper half, leaving the overall median out of both halves when n is odd.
To decide what counts as an outlier, build fences 1.5 IQRs beyond the box. Values outside the fences are plotted as separate dots, and the whiskers stop at the most extreme values that are still inside.
Worked example: a box plot with an outlier
Battery life in hours for 11 phones (made-up data), in order: 6, 8, 9, 9, 10, 11, 11, 12, 13, 14, 23.
- The median is the 6th value: 11.
- Lower half 6, 8, 9, 9, 10 → Q1 = 9. Upper half 11, 12, 13, 14, 23 → Q3 = 13.
- IQR = 13 − 9 = 4, and 1.5 × 4 = 6.
- Fences: 9 − 6 = 3 and 13 + 6 = 19. Nothing is below 3, but 23 is above 19, so 23 is an outlier.
- Draw the box from 9 to 13 with a line at 11. The lower whisker goes to 6. The upper whisker stops at 14, the largest value inside the fence, and 23 gets its own dot.
The fences are a rule of thumb, not a law. An outlier is a value to investigate: it might be a typing error, or it might be the most interesting value in the data.
Shape
When you describe a distribution, start with its shape. These four come up again and again:
Shape decides which summaries to trust. For skewed data or data with outliers, report the median and IQR; for roughly symmetric data, the mean and standard deviation work well. Unit 1 shows how skew drags the mean and gives the SOCS checklist (shape, outliers, center, spread) for describing a distribution in words. For bimodal data, no single center tells the story: describe the two groups.
Spread: range, IQR, standard deviation and variance
Two classes can have the same mean score and still be very different: in one, everyone scored close to the mean; in the other, scores ranged from very low to very high. Measures of spread put a number on that.
- Range = maximum − minimum. Quick, but it depends only on the two most extreme values.
- IQR = Q3 − Q1, the spread of the middle half. Outliers barely move it.
- Standard deviation (SD) is roughly the typical distance of a value from the mean. It uses every value, so outliers move it a lot.
- Variance is the SD squared: the average squared distance from the mean.
Worked example: standard deviation by hand
Data (n = 8, made up): 2, 4, 4, 4, 5, 5, 7, 9.
- Find the mean: 40 ÷ 8 = 5.
- Subtract the mean from each value to get the deviations, then square them (table below).
- Add the squares: 32.
- Divide by n − 1 = 7 to get the variance: s² = 32 ÷ 7 ≈ 4.57.
- Take the square root to get the standard deviation: s = √4.57 ≈ 2.14.
| x | x − 5 | (x − 5)² |
|---|---|---|
| 2 | −3 | 9 |
| 4 | −1 | 1 |
| 4 | −1 | 1 |
| 4 | −1 | 1 |
| 5 | 0 | 0 |
| 5 | 0 | 0 |
| 7 | 2 | 4 |
| 9 | 4 | 16 |
| Sum | 0 | 32 |
The deviations always add to 0, because the values above the mean balance the ones below. That's why we square them before adding.
Why n − 1? When your data are a sample from a bigger population (the usual case), dividing by n − 1 gives a better estimate of the population's spread, because a sample's values sit a little closer to their own mean than to the true one. If your data are the whole population, divide by n instead and call the results σ² and σ (“sigma”). For the 8 values above that gives σ² = 32 ÷ 8 = 4 and σ = 2. Calculators often show both, labeled s (or sx) and σ (or σx).
Why both SD and variance? The SD is in the same units as the data (hours, points), so it's easier to interpret. Variance is in squared units (hours², points²) but has neat mathematical properties that you'll use in later units.
Misleading graphs
Graphs persuade, so check them before you believe them:
- Does a bar chart's axis start at zero? Bars are compared by length, so a cut-off axis exaggerates differences. (A line graph or dot plot may reasonably zoom in, as long as the axis is labeled.)
- Do pictures scale by area? Doubling both the height and width of an icon makes it look four times as big.
- Are the intervals even? Uneven time steps or histogram bins of different widths distort the shape.
- Counts or rates? A bigger town has more of everything. Compare rates, like cases per 1,000 people, when group sizes differ.
- Is anything missing? Look for a source, labeled axes with units, and whether the time window was picked to tell one story.
The normal curve and the 68–95–99.7 rule
Many measurements, like heights of adults of the same sex, repeated measurements of the same object or scores on a large standardized test, have a histogram close to one particular smooth, symmetric bell shape: the normal curve. A normal distribution is fixed by just two numbers: its mean μ (the center) and its standard deviation σ (the spread). The curve is steepest at exactly one SD either side of the mean.
In any normal distribution:
- about 68% of values are within 1 SD of the mean,
- about 95% are within 2 SDs,
- about 99.7% are within 3 SDs.
Because the curve is symmetric, you can split these. For example, 95% within 2 SDs leaves 5% outside, 2.5% in each tail, so about 97.5% of values are below μ + 2σ.
z-scores
A z-score says how many standard deviations a value is from the mean:
Positive z is above the mean, negative is below, and z = 0 is the mean itself. Because z-scores have no units, they let you compare values from different scales.
Worked example: which test went better?
Maya scored 86 on a math test where the class mean was 70 and the SD was 8, and 84 on a Spanish test with mean 60 and SD 15 (made-up numbers).
- Math: z = (86 − 70) ÷ 8 = 2.0.
- Spanish: z = (84 − 60) ÷ 15 = 1.6.
- Her math score is further above the class, relative to the spread, even though the raw scores are close.
- If the math scores are roughly normal, the 68–95–99.7 rule puts z = 2 at about the 97.5th percentile. The exact normal table value is 97.7%.
When data are (and aren't) roughly normal
The 68–95–99.7 rule and normal percentiles only work for data that really are close to normal. Plenty of real data aren't:
- Skewed data, like incomes, house prices or wait times: a floor at zero and a long tail of large values.
- Bounded data squeezed against a limit, like scores on an easy quiz bunched near 100%.
- Mixed groups, which can give two peaks.
Numbers alone can fool you. The 40 commute times from the bin-width explorer have mean 20.5 minutes and SD 11.0 minutes. 26 of the 40 values (65%) are within 1 SD of the mean and 39 (97.5%) are within 2 SDs, close to 68% and 95%. Yet the histogram has two peaks, and the mean sits in the valley between them, where hardly any actual commutes are. A bell curve would describe these data badly.
Always graph first. Before using normal methods, draw a histogram or dot plot and ask: one peak? Roughly symmetric? No outliers or hard edges? If not, describe the data with the median and IQR instead.
Common mistakes
Drawing a histogram like a bar chart. Histogram bins are number ranges in order, so the bars touch. Gaps belong only where a bin really has a count of 0.
Trusting one bin width. A single histogram can hide two peaks or invent bumps. Try a narrower and a wider width before you describe the shape.
Running the whisker to the outlier. When a value is beyond a fence, the whisker stops at the most extreme value inside the fence and the outlier gets its own dot.
Forgetting to square, or to take the root. Raw deviations always add to 0. Square them, add, divide by n − 1, then take the square root for the SD. Stopping before the root gives the variance.
Mixing up the denominators. Divide by n − 1 for a sample (s) and by n for a whole population (σ). Check which your calculator is showing.
Using 68–95–99.7 on skewed data. The rule is for roughly normal data only. Graph the data first.
Getting the z-score sign backwards. z = (x − mean) ÷ SD. A value below the mean has a negative z-score.
Practice
People behind the ideas
1733 Abraham de Moivre circulates a short Latin paper on approximating the terms of (a + b)n for large n. His approximation is an early form of what we now call the normal curve, though he saw it only as a shortcut for binomial calculations. It is published more widely in 1738, in the second edition of his book The Doctrine of Chances.
1809 Carl Friedrich Gauss, working out the orbits of planets and comets, shows that if measurement errors follow the normal law, the average of the measurements is the most probable value. This ties the curve to his method of least squares.
1835 Adolphe Quetelet's Sur l'homme describes the “average man”, with human traits spread around the mean along the normal curve: the first use of the curve for something other than errors.
1891 Karl Pearson uses the word “histogram” in a lecture at Gresham College in London, in a series on “The Geometry of Statistics”. In print in 1895, he explains it as columns whose areas show the frequencies.
1893 In a lecture on 31 January, Pearson introduces the term “standard deviation” as a replacement for “root mean square error”. It is in print by 1894.
1977 John Tukey's book Exploratory Data Analysis is published. Its box-and-whisker plot dates to around 1970, when it appeared in the book's preliminary edition. His stem-and-leaf display had appeared in print in 1972.
Quiz
Ten questions. Pick an answer for each, then check.
Talk about it
- A news site shows a bar chart of two candidates' poll numbers with the axis starting at 40%. Who might want the graph drawn that way, and why?
- Two classes both average 75 on a test, but one has an SD of 4 and the other an SD of 15. Describe what each class's results might look like. Which class would you rather teach, and why?
- Quetelet imagined an “average man”. Can a real person be average in every measurement at once? What goes wrong with designing, say, a chair or a cockpit for the average?
- Why might a histogram of the ages of everyone at a school (students and staff) be bimodal? Think of another data set that would have two peaks.
Sources
- Jeff Miller, Earliest Known Uses of Some of the Words of Mathematics, H entries: “histogram”, coined by Karl Pearson in a November 1891 lecture in his Gresham College series “The Geometry of Statistics”, and his footnote in the Philosophical Transactions (A) 186 (1895), p. 399.
- Jeff Miller, Earliest Known Uses, S entries: “standard deviation”, introduced by Pearson in a lecture of 31 January 1893 in place of “root mean square error” and in print in Philosophical Transactions (A) 185 (1894); “stem-and-leaf”, from Tukey's “Some Graphic and Semigraphic Displays” (1972).
- Wikipedia, Box plot: the box-and-whisker plot introduced by John Tukey in 1970 and published in Exploratory Data Analysis (1977), citing Hadley Wickham and Lisa Stryjewski, “40 years of boxplots”.
- MacTutor History of Mathematics, Adolphe Quetelet biography: the first to use the normal curve other than as an error law, and the average man in Sur l'homme.
- Wikipedia, Adolphe Quetelet: Sur l'homme et le développement de ses facultés published in 1835.
- Wikipedia, Normal distribution: history: de Moivre's 1733 paper and the 1738 second edition of The Doctrine of Chances; Gauss (1809, Theoria motus corporum coelestium) linking the normal law of errors to the arithmetic mean.
- Normal-curve areas (68.27%, 95.45%, 99.73%, 97.72%, 81.86%) were computed from the standard normal distribution function; the explorer uses a numerical approximation accurate to better than 0.0001%.
- All data sets on this page (commute times, quiz scores, battery life, test scores and brand sales) are made up for teaching.