Advanced Placement courses have become so associated with college readiness that the number of classes offered in a high school is considered a bellwether of the school’s seriousness about college readiness and access to higher education. A new study suggests, however, that Advanced Placement’s benefit may come from the placement-style assessment, rather than the course material.
In a study in the latest issue of The Journal of Educational Research, Utah Valley University researchers led by psychologist Russell T. Warne analyzed the records of more than 90,000 students who graduated from Utah public schools in 2010 and 2011. They divided the students into four groups: those who did not take an AP English or Calculus class; those who took one of the courses but not the test; those who took the courses and tests but scored only a 1 or 2; and those who took the courses and passed the tests with at least a 3—the minimum generally needed for college credit.
After controlling for 70 background characteristics—including students’ ethnicity and socioeconomic background, cumulative high school grade point average, and whether they were classified as migrant students or had disabilities—the researchers analyzed how students performed on the ACT college entrance exam. (The College Board, which oversees Advanced Placement, has a competing entrance exam, the SAT.)
Based on the 2010 cohort, the authors found that students who took challenging AP coursework without taking the test performed about the same on the composite ACT as those who did not participate in Advanced Placement, 21.4 out of 36. That’s slightly above the nation’s average composite ACT score in 2014, which is 21. Students who took the class and test but failed the end-of-course test scored a quarter to a half of a point higher—"similar to the increase in ACT scores for students who experience traditional test-coaching methods in preparation for college admissions exams,” the authors wrote. The 2011 cohort had similar results, but did not have as representative a sample of students.
By contrast, students who took and passed AP English scored about 3 to 4 points higher on the ACT composite, and those who took and passed AP calculus had 1-to 3-points-higher ACT composite scores. That’s a significant improvement, but how significant would depend in part on where the student was applying after high school; moving from a composite of 20 to one of 23 would put a student in the top one-third of scores and in better contention for selective universities, but many academic scholarships start at scores of 25 or 27.
“There’s a lot of reasons to take AP classes with no intention to take the test: Your friends are taking the class; your mom makes you take it; you get an automatic boost to your GPA,” Warne said. “Just merely enrolling in these classes isn’t the answer.”
If their secondary schools did not lay the groundwork for the challenging coursework in AP classes, the authors concluded, “it is likely that many students in American high schools would have little chance of passing an AP exam. For these students, merely enrolling in the course may not be the best use of their and their school’s time, money, and resources.”
The study did not draw any conclusions about why AP benefits seemed to go primarily to those who performed well enough for college credit—though the findings are in line with other studies on the program. After puzzling over the results for a year, Warne told me, “There’s nothing magical about the test. Paying the money to take an AP test is not going to make you smarter. The best explanation is the studying for [the AP test], the effort, preparing ... I think that’s the key. That’s not the only explanation, but to me that’s the most plausible.”
The study does not capture other potential benefits from taking advanced courses, such as exposure to college-level material or simply a more academically focused classroom environment, he said. “The most important thing is that we be realistic about our expectations of the AP program. It’s not a silver bullet; there’s no evidence giving it to your school will turn a failing school to a good school. For students thinking about college, it can be a very good program.”
Many states require high-stakes exams to graduate, but, Warne added, “a lot of high-stakes tests in high schools are not challenging to the students that AP is challenging for.” Moreover, students can’t opt out of taking an exit exam in high school unless they leave school altogether. Wayne said he next wants to examine which of the students who are taking AP classes decide to take the test, and how they make that decision.
Recently some schools have questioned whether the courses really contain college-level content, but the Utah study did not include an analysis of any differences in the content students took through the AP classes.
You can see more details on the study in the video below:
I’ll update with more perspective from the College Board on what might be going on here.
A version of this news article first appeared in the Inside School Research blog.