In their Commentary “The Diogenes Factor,” (April 8, 1998), researchers Herbert J. Walberg and Rebecca C. Greenberg make four unusual assertions, any one of which would justify a much more detailed response than the following review of those assertions with my proposed resolution.
The first and cardinal assertion concerns, in the authors’ words, the “pervasiveness of what we will call ‘the Diogenes factor’ in program evaluation.” Diogenes was a 4th century B.C. Greek philosopher, and the most famous of a group known as the Cynics. He doubted everything, and the integrity of everyone. To dramatize his point, Diogenes walked the streets of Athens by day, carrying a lit lamp, in search of “an honest man.”
| Skepticism, whether about the value of penicillin in the early part of this century or school improvement today, is a hallmark of science. | 
|
As when reviewing the Jones-Gottfredsons paper, Mr. Walberg and Ms. Greenberg fail to mention some of Mr. Venezky’s central conclusions. He states, for example, that “SFA students who remain continually in SFA schools showed significant advantages over control-school students in achievement and in avoiding grade-level retention and special education referral.” Mr. Walberg and Ms. Greenberg conclude that “the Success for All developers and independent reviewers differ hugely in their estimates of its effects.” Clearly, their statement is at variance with the facts. At some point, readers must regard Diogenes’ lamp as being double-edged.
Mr. Walberg and Ms. Greenberg presented four additional “reasons [that] probably account for ... differences” between Success for All and control schools in the various Success for All studies.
1. They speculate that because Success for All insists that a supermajority of the teachers vote to accept the program, the experimental sites are “hardly run-of-the-mill schools.” In fact, many school reform efforts insist on majority or “supermajority” votes of teachers prior to implementation. Yet, as noted in the Special Strategies Studies and in the Memphis Restructuring Initiative (Steven Ross, William Sanders, Paul Wright, and Samuel C. Stringfield, 1998), in proactive, controlled quasi-experiments examining diverse reform designs, some of these designs’ implementations have not resulted in measurable achievement gains.
In Memphis, longitudinal gain-score analyses from the year prior to implementation indicated that the schools had previously been producing less achievement gain than matched controls. The most plausible interpretation of these data is that some of the reforms, including but not limited to Success for All, helped “the mill” run better.
2. They state that “Success for All concentrates on reading, possibly sacrificing math, science, and other subjects.” In fact, in both Special Strategies and the Memphis Restructuring Initiative, Success for All and a few other designs produced gains across multiple achievement domains. In Special Strategies, the gains were in both reading comprehension and mathematics concepts and applications. In Memphis, the achievement-gain comparisons between restructuring schools and control/other schools were “generally consistent across academic subjects.”
3. They charge that Success for All uses tests that are “unlike standardized national achievement tests used by independent evaluators.” In fact, both the Special Strategies and Memphis Restructuring studies relied on nationally normed achievement tests, not measures devised for or chosen by one reform design. Yet both studies found significant, multi-year, positive effects for Success for All, and for some other programs.
4. They accuse Mr. Slavin and his colleague Olatokunbo Fashola of citing their research’s “own positive effects, not its own negative findings, nor the negative findings of independent evaluators.” In fact, Mr. Venezky concluded that Success for All had positive effects. The Jones-Gottfredsons research made clear that theirs was hardly a typical implementation. Other local and national evaluations across a diverse spectrum have found positive effects for Success for All, and for some other designs.
In their final assertion, Mr. Walberg and Ms. Greenberg claim that Title I and similar programs designed to help America’s less fortunate children have “failed” and “are worse than doing nothing.” They don’t name the “five independent evaluations” of Chapter 1 that they claim “showed little achievement difference between program and control groups,” so the assertion cannot be independently evaluated.
The two most recent reviews of evaluations of the Title I/Chapter 1 program of which I’m aware are a meta-analysis by Geoffrey D. Borman and Jerome V. D’Agostino (1996) and a review by Edward L. McDill and Gary Natriello (in press). Both sets of authors reviewed many more than five studies, both named their sources, and both found positive, increasing effects of Title I. Positive effects are not failures, and certainly are not “worse than doing nothing.”
The need to raise the academic achievements of all students, and particularly those of children born into situations that place them at risk of academic failure, has never been greater. “Big muscle, small skill” jobs are vanishing across the nation. The income differential between young male college graduates and high school dropouts, below 40 percent a generation ago, is now over 200 percent.
Yale University’s Dr. James P. Comer, Stanford University’s Henry Levin, Ohio State University’s Gay Sue Pinnell, Johns Hopkins University’s Robert Slavin, and the developers of the New American Schools designs are among the leaders of the current generation of academic-improvement-design developers. Their task is daunting. They deserve our encouragement. Their research, like all research, invites our appropriate skepticism, but not paralyzing, defeatist cynicism.
We need a great deal more longitudinal research on promising programs. Those studies will necessarily be difficult and expensive. In this area, medical research can be instructive. As is the case in medicine, most of that funding will necessarily be federal. For a fraction of the budget of the National Institutes of Health and the Centers for Disease Control and Prevention, the science of educational reform could make dramatic progress. That progress could greatly inform and enhance the effectiveness of all of American education, including the Title I program.
Until such evaluations are funded and conducted, however, we have to make intelligent assessments of the quality and consistency of the evidence we have. Declaring all evidence to be so flawed and all funders and researchers suspect does not move educational reform forward. Rather, such cynicism paralyzes.
Our children can learn more. Our schools can help children learn more today, and our schools can improve over time. The best available evidence is that some of the current generation of reforms can help. Science, not cynicism, will help us help our children. They deserve no less than scientific progress.
Samuel C. Stringfield is a principal research scientist at the Center for Social Organization of Schools at Johns Hopkins University in Baltimore.
