School-based reward programs that offer students such incentives as cash, free MP3 players, or other gifts appear to produce improved reading achievement across grade levels, preliminary findings from an ongoing research project suggest.
The analysis, which looked only at charter schools because of the prevalence of incentive programs in the independent public schools, found no impact on students’ performance in mathematics.
Released today by the Center for Research on Education Outcomes, at Stanford University, the study comes as a growing number of school districts and charter schools around the country are experimenting with such reward programs in the hope of improving student learning and behavior. The analysis suggests that incentive programs may well be a cost-effective measure to help raise achievement.
“It’s not a silver bullet, but for very little investment, you seem to get a pretty consistent bump,” Margaret E. Raymond, the director of the Stanford center and the study’s author, said in an interview.
The study found an average gain of 4 percentile points in students’ performance on standardized state reading tests for each year they participated in a rewards program. So, Ms. Raymond said by way of illustration, an elementary student who started out at the 50th percentile in reading could be expected to move up to the 65th percentile after several years in a school that uses an incentive system. The gains would be in addition to academic improvements the student would typically be expected to make.
“While this analysis is exploratory and based on a small number of cases, it shows consistent impact of incentive and reward programs across grades and designs in terms of student-achievement gains on state achievement tests in reading,” Ms. Raymond writes in an executive summary of the study.
Reward Systems Vary
The study was based on a survey of 250 charter schools. Of the 186 respondents, 106, or 57 percent, reported that they use a system of rewards and incentives with their students. Most of the systems combined student behavior, academic effort, and academic results in their reward structures, the report said.
A smaller subset of 47 schools was used for the academic analysis. The analysis focused on charter schools, the report says, because student-incentive programs have become a “widespread practice,” owing to such schools’ operational flexibility.
Across the incentive systems, classroom behavior was rated in 93 percent of the schools, followed by completion of assignments in 89 percent. Student-adult interactions, such as speaking respectfully or accepting direction,were rated in 86 percent of the schools.
The analysis looked at a variety of rewards, such as obtaining “access to select activities,” receiving certificates of merit, getting credit to purchase items at the school store, and receiving cash, a college-fund contribution, or other rewards.
It found stronger gains in schools where the adults share a common belief in the value of such incentive programs.
“When we look inside the black box,” Ms. Raymond said, “it turns out that having all the adults in the school on the same page makes a big difference.”
Caution Urged
Eric P. Bettinger, an associate professor of economics at Case Western Reserve University in Cleveland, who recently published his own study on performance incentives in one school district, said that while Ms. Raymond’s research holds considerable potential to shed more light on the impact of incentive programs, its academic results to date should be interpreted with caution. (“Students in Cash-Incentives Study Score Higher in Math,” Feb. 27, 2008.)
Mr. Bettinger pointed to a caveat noted in the study that schools that adopt reward programs “may be systematically different in some unmeasured way from those that do not use them,” as the report’s executive summary puts it. If that were the case, that “selection bias” could skew the results.
“In this paper, the clear worry is that the selection bias may be driving the results,” Mr. Bettinger said. “As she gathers more data,” he said of Ms. Raymond, “we’ll have a much clearer picture of how much is an achievement effect versus how much is driven by confounding factors.”
To test for selection bias, the analysis did try to take into account the increased tendency of schools to have rewards systems if they are located in neighborhoods in which high numbers of adults lack a high school diploma. When that factor was considered, the apparent impact of the reward systems disappeared.
But the study emphasizes that the sample of schools was too small to draw strong conclusions at this point, and that the addition of more data and more schools would help interpret the achievement results with more confidence.
“We do not have any certainty that there is selection bias going on,” Ms. Raymond said. “You can never prove there is selection bias.”
Mr. Bettinger said he was pleased that the study’s survey had gathered information on the various types of incentives schools use. Yet he pointed out that the study does not differentiate which incentives may have the strongest effects, as the programs are lumped together for the academic analysis.
“I hope [Ms. Raymond] can really separate the various types of incentive programs” in future research, he said. “All the incentive programs are very, very different in nature.”
Ms. Raymond said she hopes in later research to provide such differentiated analysis.
