All articles / SAT Math / Problem-Solving & Data Analysis
Evaluating Statistical Claims for the SAT: Complete Study Guide + Free Practice Problems
Everything the SAT tests about observational studies and experiments in one place: random sampling, random assignment, bias, confounding variables, and what conclusions a study can and cannot support — with worked examples and a free practice quiz.
Practice this quiz
Free · No signupEvaluating Statistical Claims
Observational studies, experiments, sampling, and valid conclusions.
What does "evaluating a claim" mean?
This SAT topic doesn't ask you to calculate anything — it asks you to read a short description of a study and judge what conclusions it actually supports. The SAT is testing whether you understand the difference between a study that can show a relationship and a study that can show cause and effect, and whether a result can be generalized beyond the people who were studied.
These questions reward careful reading over calculation. Two features of the study description matter most: how the subjects were selected, and whether they were randomly assigned to different treatments.
Observational studies vs. experiments
| Observational study | Experiment | |
|---|---|---|
| What happens | Researchers observe subjects without changing anything | Researchers actively assign a treatment to subjects |
| Can show | Association / correlation only | Cause and effect, if randomly assigned |
| Example | Comparing test scores of students who already choose to study with music vs. without | Randomly assigning students to study with or without music, then comparing scores |
Only a properly randomized experiment can support a cause-and-effect claim. No matter how strong or consistent the pattern looks, an observational study can never be used to conclude that one variable directly causes a change in another.
Random sampling & generalizing results
Random sampling means every member of a population has an equal chance of being selected for the study. This is what allows the results to be generalized back to the full population the sample was drawn from — and only that population.
A researcher randomly selects 200 students from a single high school and finds most prefer online classes. Can the researcher conclude this applies to all high school students nationwide?
A random sample only lets you generalize back to the specific population it was drawn from. Random sampling fixes who you can talk about; random assignment (a separate idea) fixes whether you can talk about cause and effect.
Random assignment & establishing causation
Random assignment means subjects are randomly placed into different treatment groups, rather than choosing or being chosen for a group. This is the feature that allows an experiment to support a causal claim, because it evens out other differences between the groups.
A drug trial randomly assigns 300 volunteers to either a new medication or a placebo. The medication group shows significantly improved outcomes. Can researchers conclude the medication caused the improvement?
Bias & confounding variables
Selection bias
The sample isn't representative of the population, often because of how volunteers were recruited.
Confounding variable
An outside factor that affects both variables being studied, making it look like one caused the other.
A study finds that neighborhoods with more ice cream shops also have higher rates of drowning. Does ice cream cause drowning?
Volunteer-based samples almost always introduce selection bias, since people who choose to participate often differ systematically from those who don't. Watch for phrases like "volunteers were recruited" as a signal the sample may not be representative.
Sample size & margin of error
Larger, well-chosen samples generally produce more reliable estimates, with a smaller margin of error — the range within which the true population value is likely to fall.
A larger sample size reduces margin of error but does not fix a biased sampling method. A huge sample drawn in a biased way is still an unreliable estimate of the full population.
Common SAT traps
- Claiming causation from an observational study: only randomized experiments can support cause-and-effect conclusions.
- Overgeneralizing beyond the sampled population: results only extend to the group the sample was actually drawn from.
- Ignoring a plausible confounding variable: before accepting a causal claim, consider whether an outside factor could explain both trends.
- Assuming a large sample fixes bias: sample size and sampling method are two separate issues — a large biased sample is still biased.
Test day strategy for evaluating statistical claims
| Question signal | Fastest approach |
|---|---|
| "Can the researcher conclude that ___ caused ___" | Check for random assignment; without it, reject any causal claim |
| "Can this be generalized to ___" | Check who the sample was actually drawn from |
| Study description mentions volunteers | Flag possible selection bias |
| Two variables move together in a study | Consider whether a confounding variable explains both |
| "Which would most strengthen/weaken the study" | Look for an answer that addresses sampling method or randomization directly |
Now put it to work
One comprehensive quiz covering the full topic — observational studies, experiments, sampling, and drawing valid conclusions.
Evaluating Statistical Claims
Observational studies, experiments, sampling, and valid conclusions.