Random assignment to groups → cause-and-effect conclusions are allowed.
Random selection of the sample → results generalize to the population.
Observational study → association only.
Step-by-step strategy
Look for two things: was there random assignment? Was there random selection?
Common traps
Generalizing volunteers’ results to “all people.”
Worked example
Experiment with random assignment of volunteers: “the app likely improves sleep for people similar to those studied” (not “for everyone”).
Sample questions
Try to answer before opening the solution.
Question 1
Difficulty: easy≈ 1 min
A newspaper wants to know what fraction of a city’s residents support building a new stadium. Which of the following methods would most likely produce a biased sample?
AA. Surveying residents chosen at random from a list of all households
BB. Surveying residents selected at random from the city’s tax records
CC. Surveying every 100th name in the city’s telephone directory
DD. Surveying fans leaving a baseball game at the current stadium
Show answer and explanation
Correct answer: D. Surveying fans leaving a baseball game at the current stadium
Explanation
Fans at a game are far more likely than average residents to support a stadium, so that sample is biased.
Random selection from household lists or tax records, and systematic selection from a directory, are much more representative.
Question 2
Difficulty: medium≈ 1 min
A teacher wants to estimate the mean time her 150 students spend on homework each night. She asks the 25 students in her first-period class. Which population can her result most appropriately be generalized to?
AA. All students in the school
BB. All 150 of her students
CC. Students in her first-period class only
DD. All high school students in the district
Show answer and explanation
Correct answer: C. Students in her first-period class only
Explanation
The 25 students were not randomly selected from the 150; they are one intact class, which may differ from the others (time of day, course level). Without random sampling, the result describes only the students surveyed.
Generalizing to the school or district is even less justified.
Question 3
Difficulty: hard≈ 1 min
A school district compares test scores at two schools. School A’s mean score is 78 and School B’s is 84. A board member concludes that School B’s teachers are more effective. Which of the following, if true, most weakens this conclusion?
AA. School B has newer computers and larger classrooms than School A, which makes teaching there easier.
BB. School B admits students by an entrance exam, while School A admits all students in its neighborhood.
CC. Both schools gave the test on the same day in the same week of the school year, under identical conditions.
DD. School A has considerably more students than School B, so its teachers have less time for each one.
Show answer and explanation
Correct answer: B. School B admits students by an entrance exam, while School A admits all students in its neighborhood.
Explanation
If School B selects students by exam, its higher scores may reflect the students it admits rather than its teaching. Student selection is a confounding variable.
Newer computers might even support the teaching explanation; the test date and enrollment size do not explain the gap.