A new federal study looking at tutoring and student-transfer provisions under the federal No Child Left Behind law suggests that those policy changes are having mixed success so far in boosting students’ academic achievement in reading and mathematics .
According to the study, which tracked the progress of the 5-year-old federal program in nine large urban school districts, students who participated in the NCLB tutoring, on average, learned more in the first year of tutoring than they had in previous years. And, while those academic gains were small, they tended to grow when students stuck with their tutors for two years or more.
On the other hand, pupils who transferred to other schools under the NCLB choice provision did no better, on average, than they had in their previous schools, according to the study, which was released June 27.
The study is the first federally funded evaluation of the NCLB’s school choice and supplemental educational services (SES) provisions to examine how those programs are affecting student achievement, rather than how or whether educators are implementing them.
U.S. Secretary of Education Margaret Spellings, in a speech to 245 educators and administrators attending a June 27 “national summit” here on the topic, was quick to endorse the results as good news for President Bush’s landmark education law.
“This report conclusively shows that our work—your work—is paying off,” she told conference-goers. “And, in many cases, the longer students participated, the better they did in school.”
“This is a brand-new enterprise,” she added. “It’s going to take more time and more energy and more resources for us to realize its potential.”
Whether the findings will play a role in the upcoming reauthorization of the federal program, however, is an open question, said Jack F. Jennings, the president of the Washington-based Center on Education Policy, which is also studying the NCLB law.
“This study is something to take seriously,” he said, noting that the findings, while not nationally representative, were statistically significant. “Congress isn’t about to eliminate tutoring or school choice, anyway, but this should put more of a burden on school districts to expand SES services.”
Few Takers
The new analysis, which was conducted for the department by RAND Corp., a think tank based in Santa Monica, Calif., is based on data from districts in Baltimore; Chicago; Denver; Long Beach, Calif.; Los Angeles; Palm Beach, Fla.; Philadelphia; San Diego, and Washington. Most of the achievement data analyzed came from the 2000-01 to 2004-05 school years.
In their report, the RAND researchers said they chose those districts because they have large numbers of students taking part in tutoring or school choice programs, which, for a variety of reasons, seem to be attracting few takers. The provisions only apply to schools that receive federal Title I funds because they serve high concentrations of disadvantaged students, and they don’t kick in until a school has failed to meet student-performance targets for two or more years.
Among the eligible student population nationwide, only 1 percent has elected to switch schools under the law, and 19 percent have opted for the free tutoring, according to federal statistics. Because of low response rates in some of the districts in the RAND study, the report’s achievement-related findings draw on data from just seven of the nine districts for its supplemental educational services analysis; the school-choice achievement results are based on just six districts.
The small number of students in those six districts who took advantage of the transfer option—about 3,000 in all—means that the school choice findings may not be as reliable as the findings on the SES provisions, according to the report. The SES analysis is based on data for more than 50,000 students.
“I think the main finding on the school choice side is that few students are participating,” said Ron Zimmer, the lead author of the study and a senior policy researcher for RAND. “We may have to wait until more students participate in the program before we can draw definitive conclusions.”
Results Consistent
The results for both provisions, nonetheless, were consistent with those of other studies looking at the academic impact of school choice or extra tutoring. The handful of states and districts that have evaluated the effects of the NCLB-tutoring program in their own jurisdictions have found, for instance, that the services have translated to small or negligible gains in student achievement.
While a larger body of research has examined various kinds of school choice programs other than the NCLB transfer provision, those studies have for the most part shown mixed results.
For their study, the RAND researchers used a technique called a “student fixed-effects model.” In other words, they compared the growth students made under the program to the progress they made in previous years in their regular public schools. The size of the average first-year gain they found for the tutoring services—.08 of a standard deviation—was much smaller than the kinds of effect sizes found when schools experiment with smaller classes, for example, or whole-school-improvement programs.
“But most of those tutoring programs take place over two or three hours a week,” noted Mr. Zimmer. “Maybe it’s a good thing that you can get that kind of effect in just two or three hours a week.”
Apart from the achievement findings, the study also found that both provisions were attracting the kinds of students they were intended to draw. For the most part, the students who signed up for supplemental educational services had lower achievement levels to begin with than peers who were also eligible for the program but decided not to apply. Likewise, the students who transferred to other schools tended to end up in better-performing schools than the ones they left behind.
Among other findings, the study also showed that:
• For both options, participation rates were higher among elementary school students than they were for high school students.
• Both the tutoring and school choice programs were most popular among African-American students. Regardless of students’ race, though, transfer students tended to move to schools with higher concentrations of white students than their neighborhood public schools; and
• Evidence was mixed with regard to whether private tutoring firms or district-run tutoring programs were doing a better job.
