One of the biggest early bets in the U.S. Department of Education’s Investing in Innovation program seems to be paying off: Success for All, a literacy-related, whole-school improvement model, shows signs of changing teaching practice and boosting students’ early reading skills after a year in schools.
The findings come from a new study by the New York City-based research group MDRC, the first of three installments in an ongoing $6.7 million evaluation of Success for All, a popular school-improvement model used in 1,000 schools representing 300,000 students nationwide. The program, which includes schoolwide curriculum, tutors, bimonthly student assessments, and teacher training, received $49.3 million from the federal i3 program in 2009 to expand to more schools and increase training for teachers and staff.
A year after 19 K-5 and K-6 participating schools in four states were randomly selected to launch the program in the 2011-12 school year, MDRC researchers found that kindergartners in those schools significantly outperformed demographically similar peers in a control group of 18 schools in a standardized test of phonics, the Woodcock-Johnson Word Attack. Success for All students got a boost roughly equal to 12 percent of the average annual growth for a kindergartner. Moreover, the same benefits were found for poor and minority students.
In the classroom, teachers at Success for All schools differed from those in the control-group schools in a number of ways. They were more likely, for example, to group and regroup students by ability for reading lessons—even across grades.
SOURCE: MDRC
Those benefits are in line with the learning gains found in previous studies of Success for All, which has been studied extensively since its founding in 1987, but the MDRC study “goes into more depth in relating implementation to outcomes than any study that’s come before,” said Robert E. Slavin, the chairman of the Success for All Foundation and the director of the Center for Research and Reform in Education at Johns Hopkins University in Baltimore. “It’s outstanding in giving a more detailed picture of what’s actually happening in the schools.”
Compared with teachers in schools that did not implement the program, researchers found that the teachers in the Success for All schools had more, and more varied, training in reading instruction. They later proved more likely to focus on comprehension, even in kindergarten, than teachers in control schools, and were also more likely to use cooperative-learning strategies. Also, following the Success for All design, teachers in those schools were more likely to group and regroup students across multiple grades based on their reading skills, to provide more focused instruction.
“Some of the cooperative learning that students undertake—like turning to your neighbor and telling them something about the text—are among the ways comprehension can get reinforced even with very young children,” said Janet C. Quint, an MDRC senior research associate and the study director for the evaluation project.
One School’s Story
The 700-student Central Elementary School in Allentown, Pa., is one of the schools testing Success for All as a turnaround model through i3. The school, which has repeatedly failed to meet its federal adequate yearly progress goals, met all 25 targets for the first time in 2010-11 and had more than 1,000 fewer school absences in that year. As part of the program, the school reported adding a literacy coach for three days per week, providing about 30 percent more professional development for teachers.
The evaluation report also details the challenge of implementing the whole-school program, which requires strictly scripted and paced lessons and regular assessments and regrouping of students. Surveys of teachers during the first year of implementation found many wanted clearer guidance on how to structure lessons, for example.
“I think SFA is a challenging program for teachers at the outset in many ways,” Ms. Quint said. For example, “it really is important to keep students engaged in quick lessons that move forward. It’s not critical for every student to master the lesson the first time through. I think that was challenging for some teachers who felt really uncomfortable in moving forward when some of their students weren’t getting it right away.”
Teachers and administrators also repeated long-held concerns about balancing the many moving parts of SFA’s comprehensive-school-reform model, with many schools reporting they did not have sufficient staff to provide tutors for all students who needed them or put in place the school committees needed to implement the program’s whole-schools reforms. Similar complaints about comprehensive school reform programs stymied previous federal efforts to expand such programs in the late 1990s.
The complexity of the program may partly explain why Success For All has not been keeping pace with its scale-up targets under i3: The group initially proposed expanding its whole-school program to 1,100 schools in five years, 550 of which would receive startup support via the i3 grant. (Central Elementary was one of these.)
Now, Mr. Slavin said Success for All will be lucky to recruit half that many new schools for expansion during the duration of the i3 grant, and all, not half, of them will receive the startup money.
“The economy has been so awful, schools have been struggling just to keep their staff, not to mention taking on any kind of reform program,” Mr. Slavin said. “We expected to have a real rush of schools interested in signing up, particularly with the i3 incentives, but ... that hasn’t happened. We’ve had to do some real marketing.”
More Studies Expected
Still, researchers will continue to study students in the first group of expansion schools as they progress through elementary school. Two additional studies will look more broadly at whole-school changes, as well as longitudinal progress for 2nd graders and older students. These will also include comprehension skills, which Ms. Quint said are more difficult to test in early grades.
MDRC’s report is the first of the evaluations from the initial cohort of four programs that won the largest grants from i3, which are based on prior evidence of effectiveness.
The Knowledge Is Power Foundation, Ohio State University’s Reading Recovery program, and Teach For America all received scale-up grants of about $45 million to $50 million each, and all are on track to meet the rigorous evaluation standards set by the federal What Works Clearinghouse. However, final evaluations for the programs likely will not come until the end of the grants or even a few years after, based on the average length of large-scale experimental studies. For example, Steve Mancini, the spokesman for KIPP, said an initial descriptive report by its evaluator, the Princeton, N.J.-based Mathematica Policy Research, will be out later this year, but it will not focus on academic progress data as the Success for All report did; rather, academic results will be evaluated in a final evaluation at the end of the grant period.
“One of the principles behind i3 was that the grants should be scaled to what we know about what works,” Ms. Quint said. “I think that at least these early SFA impacts suggest that this approach makes sense; you want to scale up things that have been shown to work and continue to study them to ensure they are still effective reforms.”
