A study released last week suggesting that California’s high school exit exams are affecting some student demographic groups more than others is the latest in a small spate of studies pointing to trade-offs from policies that require high school students to pass state tests to graduate.
Twenty-six states have exit exams in place or will by 2012, according to the Center on Education Policy, a Washington-based group that tracks accountability policies.
While proponents see the exams as a way to spur students to higher levels of achievement, critics worry that the requirements come down harder on students from poor families, minority groups, or underresourced schools.
The California study, which was released April 22 by the Institute for Research on Education Policy and Practice at Stanford University, gauges the effect of the Golden State’s 6-year-old graduation policy on the first three graduating classes to take the new exit exams in four of the state’s largest districts. Collectively enrolling 110,000 high school students, the districts serve students in Fresno, Long Beach, San Diego, and San Francisco.
Twenty-six states have exit exams in place or will by 2012, according to the Center on Education Policy, a Washington-based group that tracks accountability policies.
SOURCE: Center on Education Policy, May 2008
Researchers found that, after 2004, when 10th graders took the exit exams for the first time, graduation rates across the four districts declined by 3.6 to 4.5 percentage points each year.
During the same time period, student achievement, as measured by other state tests that the students take in 11th grade, did not significantly improve.
The detrimental effects of the new policy were harder on girls in the bottom achievement quartile than on boys. Girls experienced a 19-percentage point drop in graduation rates after the California High School Exit Exam, or CAHSEE, was implemented, while the graduation rate for boys with similar academic profiles decreased by 12 percentage points over the same period.
Likewise, graduation rates among the poorest-performing black, Hispanic, and Asian-American students declined by 15 to 19 percentage points following the enactment of the exit-exam policy. The comparable graduation-rate drop for white students in the same achievement quartile was 1 percentage point.
“These are clearly troubling, and no one can be happy with a policy that is having such disproportionate effects,” said Sean F. Reardon, an associate professor of education at Stanford who led the study.
Effects of Failing
Results for other working papers that are being circulated in academic and education policy circles suggest somewhat similar stories in other states.
In Massachusetts, for instance, a group of Harvard University researchers, in a study looking just at students who barely passed or barely failed that state’s exit exam in 10th grade, found that being labeled a failure can have a detrimental effect on low-income students in urban schools.
Even though students have plenty of opportunities to retake the exam—and most do—poor, inner-city students who just missed the passing cutoff in 10th grade are 8 percentage points less likely to graduate on time than demographically similar students who just barely passed, even though both groups scored at roughly the same levels on the 10th grade exam. Failing or passing the tests seems to have no statistically significant effect, though, on the probability of graduation for wealthier, suburban students.
And in New Jersey, which in the 1980s became one of the first states to require students to pass a statewide assessment to get a diploma, a not-yet-published study that also looks at students at the pass-fail margin shows a similar result. It finds that barely failing the test decreases the likelihood of graduation for students overall and especially so for black, Hispanic, and low-income students.
“I’m very sympathetic to the argument that we need to convey and find ways to enforce high expectations for students,” said Thomas S. Dee, an economist at Swarthmore College, in Swarthmore, Pa., who has also studied what happens in states across the nation when they enact graduation-exam requirements. He is not connected to the California, Massachusetts, and New Jersey studies.
“But I’m uncomfortable with this form of doing it, because it targets very strong penalties on the most at-risk students,” Mr. Dee continued. “The pejorative consequences appear to be concentrated in populations and communities that lack the capacity to meet these standards.”
But Jay P. Greene, another expert who has studied the effects of exit-exam policies across states, said apparent test-score disparities may not be reason enough to dismantle such policies.
“It’s sort of like blaming the thermometer for fevers,” said Mr. Greene, a professor of education reform at the University of Arkansas at Fayetteville. “There are differential passing rates for racial and ethnic groups, but it doesn’t speak about the test. It speaks about different educational opportunities and differing levels of social change.”
“The easiest way to maximize the graduation rate and ensure there is no differential impact is to give everyone a diploma at birth,” he said. School systems don’t do that, he added, because their diplomas are intended to signal that students possess certain skills that are needed to succeed.
Benefits vs. Costs
In California, state schools Superintendent Jack O’Connell, agreed with Mr. Greene.
“I continue to believe that the exit exam plays an important role in our work to ensure that a high school diploma has meaning,” he said in a statement. “I believe that the biggest mistake we could make is to view this report as a reason to lower our expectations for any student.”
While California’s effort did not seem to spur any achievement gains, the lead author of the Massachusetts study noted that achievement scores in that state, as measured by the National Assessment of Educational Progress, have risen in the 11 years since the state began administering its rigorous Massachusetts Comprehensive Assessment System, or MCAS, tests. Passing the tests became a graduation requirement with the class of 2003.
“But there’s no evidence on whether that’s the result of the exit exam or more money being put into the system,” said John P. Papay, a graduate student in education at Harvard and the primary author of that study.
At this point in the research process, the Harvard researchers are also hard-pressed to say whether the differential effects they found among Massachusetts students at the passing margin were due to the positive effects of passing the test or the negative effects of failing.
“Passing might give students more confidence,” Mr. Papay added, “but failing might make them more discouraged.”
Likewise, theories vary as to why exit-exam requirements seem to affect different groups of students differently. The usual explanation is that disadvantaged students often attend schools with fewer resources, greater challenges, and less-credentialed teachers than the schools that enroll more-advantaged students. And students from affluent families, many of whom typically are white, may have better access to tutors and other kinds of supports to enable them to pass their exams on another try.
In the California study, though, those kinds of explanations didn’t fit the data. For one, the researchers could find no effect for the policy change on low-income students, only for students of color and female students. The researchers also tried to test the idea that differences among schools were to blame by analyzing data for students within the same schools, but they got the same pattern of results.
That led the research team to advance a new explanation for the differences in effects: stereotype threat. The term refers to the tendency of people to fare less well on tests when they fear their efforts will confirm a negative stereotype about their group.
Psychological Threats
It was first coined by the psychologists Claude M. Steele and Joshua Aronson, who showed in a 1995 experiment that African-American students performed more poorly than white students when their race was emphasized in some way.
More than 300 experiments since then have demonstrated the same tendency among women, Asian-Americans, and even a group of white men taking a math test alongside Asian-American students.
The California researchers said stereotype threat became a likely explanation after they looked at students’ past scores on state tests. The data suggested that black students, girls, Latino students, and Asian-Americans had all underperformed on the exit exam in ways that could be deemed characteristically stereotypical. Asian-American students, for instance, turned in lower-than-predicted performances on the English section. Girls underperformed in math.
“It’s a very specific pattern, so it’s hard to explain based on effort,” said Mr. Reardon. “That’s what persuaded me that we have this stereotype-threat story going on, that we have this other set of tests to compare it to, and they don’t show the same pattern.”
Mr. Reardon said a large body of research points to classroom interventions to lessen perceived psychological threats for students. Schools might also provide additional instruction to low-achieving students or explore new ways to hold schools accountable for student achievement, he suggested.
Russell W. Rumberger, a professor of educational leadership and organization at the University of California, Santa Barbara, said growing numbers of states, including Indiana, Massachusetts, and Texas, are also exploring requiring students to pass end-of-course tests in individual subjects, rather than broader exit exams, as a potentially fairer way to boost graduation rates for some students without diminishing the value of a diploma. (“Exit Scramble,” Aug. 13, 2008.)
“The cynic in me worries that we’re just going to continue to see these policies proliferate, because it seems like an obvious way to convey the expectations that we should have for students,” Mr. Dee said, “and the negative effects appear to be hidden from public discussion.”
