Corrected: An earlier version of the story misquoted how Mr. Glazerman characterized the study. He said the effects were “precise,” not “robust.”
Teachers who received two years of comprehensive induction services boosted student scores in reading and math more than teachers in a comparison group who didn’t receive the support, a study released today by the U.S. Department of Education’s Institute of Education Sciences finds.
But the induction services didn’t make teachers more likely to stay in their schools, districts, or the profession—nor were they any more likely to report feeling prepared, it concludes.
The findings represent the third and final year of results from a randomized experiment focusing on the impact of intensive mentoring programs.
Conducted by Mathematica Policy Research, a Princeton, N.J.-based evaluation firm, the experiment compares outcomes for teachers who received comprehensive induction provided by trained mentors with those who received typical novice-teacher supports provided by their district.
The student-achievement findings stand in contrast to those from the first two years of the study, which indicated no effects on scores. (“Intensive Induction Shows Little Impact,” Nov. 5, 2008.)
The data don’t provide insights into what may have produced the new pattern.
“We don’t really have enough strong evidence from the quantitative analysis to support one story or another,” said Steven M. Glazerman, a principal researcher at Mathematica.
Delayed Effects?
Comprehensive programs take a more-structured approach to new-teacher support and include a careful selection of teacher mentors, formative assessments to gauge teacher progress, and release time for mentors to observe their charges and provide feedback on their instruction.
To study the programs’ effects, the researchers assigned a group of more than 1,000 teachers across 17 districts to either a treatment group receiving intensive mentoring services provided by the New Teacher Center or by Educational Testing Service, two nonprofit providers located in Santa Cruz, Calif., and Princeton, N.J., respectively; or to a control group, in which they received whatever services were offered by the district.
All the teachers in the treatment group received one year of the services, and teachers in a subset of seven of the districts received two full years of the services.
Earlier reports found that “treatment” teachers, while receiving the services, were more likely to report that they had a mentor and spent more time overall in mentoring activities than their peers. But those activities didn’t contribute to higher rates of student achievement or retention.
The most recent report upends that pattern. In the third year of study, induction programs led to statistically significant improvements on student test scores in both reading and mathematics.
The effect sizes, the report states, are large enough to boost a student scoring at the 50th percentile in both subjects to the 54th percentile in reading and the 58th percentile in math. Such increases are especially noteworthy because they appeared a year after teachers in the subset had stopped receiving the specialized support.
The sample sizes for the student-achievement component of the study were small, measuring the gains of students of only 74 teachers in reading and 68 in math. Small sample sizes can be problematic in quantitative analyses, but the researchers found that those estimates were the most precise, Mr. Glazerman said.
The report notes, however, that looking at the data through other lenses—for example, not accounting for students’ prior achievement—led to lower estimates of the impact of the mentoring.
The report does not find any evidence, however, that the comprehensive induction programs increased the likelihood that those teachers would stay in their districts or in the teaching profession. And those teachers did not report feeling significantly more prepared to instruct.
Reaction to the findings from the field was, in general, cautiously optimistic.
Officials from the New Teacher Center had criticized earlier reports generated by the study, contending that its overall implementation has been problematic. They cited problems in selecting mentors and delays in assigning them to teachers.
The director of policy for the center, Liam Goldrick, deemed the new positive findings “encouraging.” But the center continues to believe that the study’s implementation may have skewed the results.
“We feel the effects could even have been stronger if those limitations hadn’t been present,” he said. “For us, the power of the work came through despite the presence of some of these real obstacles.”
