Opinion
School & District Management Opinion

“Take Aways” from Two Months of What Works Clearinghouse Reviews

September 15, 2007 5 min read

For the most part public schools do not develop their own curriculum or pedagogy - they buy them, primarily from private enterprise, and mostly from for-profits. I find it ironic that: 1) outsourcing the most central of public education’s core teaching and learning functions is never part of the “privatization” debate; and 2) the quality of those products services and programs didn’t become a subject of serious interest until No Child Left Behind. Yet “god forbid” we allow private enterprise to operate public schools under charter or contract, or manage teachers’ professional development. Given the discrepancy, it’s hard to believe that debate has much to do with what kids need rather than what adults want.

Speaking of quality, and the Reading First fiasco notwithstanding, program efficacy is supposed to matter today. Educators, indeed taxpayers, have a desperate need for independent reviews of program efficacy. In their absence, marketing dominates the sales race.
After a very slow start, the What Works Clearinghouse (WWC), a Department of Education Institute for Education Sciences’ activity (contracted out to a consortium of evaluation organizations) to examine the evidence providers use to claim their programs are grounded in “Scientifically Based Research” (SBR), is starting to pump out reviews across a range of subjects, grades and providers.

The sixteen reviews covered below happens to coincide with September’s two-month catch-up issue of School Improvement Industry Research and Evaluation Announcements http://www.k-12network.com(we vacation in August). There’s something to be learned from the list:

August 13, 2007:

Earobics® - The WWC found Earobics® to have positive effects on alphabetics and no discernible effects on fluency.

Ladders to Literacy for Kindergarten Student… .potentially positive effects on alphabetics and fluency and mixed effects on comprehension.

Success for All…. potentially positive effects on alphabetics and general reading achievement and mixed effects on comprehension.

Voyager Universal Literacy System…. potentially positive effects on alphabetics and potentially negative effects on comprehension.

July 30, 2007:

Doors to Discovery…. no discernible effects on oral language, print knowledge, and phonological processing.

Let’s Begin with the Letter People®…. no discernible effects on oral language and potentially positive effects on print knowledge and phonological processing.

Pre-K Mathematics - The WWC found Pre-K Mathematics combined with DLM Early Childhood Express software to have positive effects on mathematics achievement.

Project GRAD…. no discernible effects on progressing in school or on completing school.

Waterford Early Reading Level One…. no discernible effects on oral language or on print knowledge.

July 23, 2007:

SRA Real Math Building Blocks PreK…. positive effects on mathematics achievement.

July 16, 2007:

Cooperative Integrated Reading and Composition® (CIRC)…. no discernible effects for comprehension.

Peer-Assisted Learning Strategies (PALS)©…. potentially positive effects on alphabetics, fluency, and comprehension.

Beginning Reading: Read Naturally….- no discernible effects on fluency and reading comprehension.

Beginning Reading: Waterford Early Reading Program…. potentially positive effects on alphabetics and no discernible effects on comprehension.

Talent Development High Schools…. no discernible effects on progressing in school or completing school.

Early Childhood Education: Literacy Express…. positive effects on print knowledge and phonological processing, potentially positive effects on oral language and math, and no discernible effects on cognition.

Observations

1. If we take this list and divide it into a crude buyers guide figure according to programs that have some positive evidence and those that don’t, and providers who are nonprofit or for profit, here’s what we get:

Some Positive Finding

For-Profit
• Earobics
• Voyager Literacy
• Pre-K Mathematics
• Waterford Early Reading Program™

Nonprofit
• Success for All
• Ladders to Literacy
• PALS
• Literacy Express

No Positive Finding

For-Profit
• Doors to Discovery
• Let’s Begin with the Letter People
• SRA Real Math
• Read Naturally
• Waterford Early Reading Level One

Nonprofit
• Cooperative Integrated Reading and Composition
• Project GRAD
• Talent Development High Schools

There’s no great quality advantage tied to the label “nonprofit” or any great black mark to “for profit.” Both groups have programs with and without evidence of efficacy. The same for-profit provider can have programs with and without evidence – take Waterford, So can the same nonproit - Cooperative Integrated Reading and Composition is offered by the Success for All Foundation..

2. Our measures of program efficacy are crude and imperfect, but they are still a far better guide than nothing - or marketing brochures. The question now is whether Congress should allow states to use federal funds to buy programs that can’t jump this bar, when there does seem to be a reasonable range of choice. In short, if you can’t pass WWC review with at least some positive finding, can your program really be said to rest on scientifically based research, and should taxpayers be subsidizing a school’s data-free decision to buy it?

3. There’s some kind of cost-benefit analysis that needs to be done by educators before any for any purchase. Still, at any given price what matters to kids is outcomes, not whether the provider has a cash surplus after paying out expenses to deliver the program, or whether that surplus is retained by a nonprofit or passed along to a company’s owners.

4. Jim Ketelsen of Project GRAD and Jim McPartland of Talent Development were no less celebrated a few years ago than John Schnur of New Leaders for New Schools is today, and given the unfairness of scheduling could just as easily taken his place at the recent Gang of Forty-Four hearings on NCLB. And, as capable a manager as John is (and I admire him greatly), he is even less able to muster enough evidence of the efficacy of New Leaders for New Schools to meet WWC’s low bar. If Congress is really serious about the “supply side” of school reform, it has to turn its focus away from philanthropy and its never-ending array of grant-driven pilots unsullied by evaluation. It has to get serious about SBR and about the fee-driven for- and nonprofit providers offering the vast bulk of products and services aimed school improvement who are in this business for the long haul – and the vast amount of potential private sector capital that isn’t flowing to them. Congress has got to start thinking about how to stimulate this industry around “what works”, and only what works..

Related Tags:

The opinions expressed in edbizbuzz are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.

Events

Student Well-Being Webinar Boosting Teacher and Student Motivation During the Pandemic: What It Takes
Join Alyson Klein and her expert guests for practical tips and discussion on how to keep students and teachers motivated as the pandemic drags on.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
A Holistic Approach to Social-Emotional Learning
Register to learn about the components and benefits of holistically implemented SEL.
Content provided by Committee for Children
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
How Principals Can Support Student Well-Being During COVID
Join this webinar for tips on how to support and prioritize student health and well-being during COVID.
Content provided by Unruly Studios

EdWeek Top School Jobs

Interdisciplinary STEAM Specialist
Smyrna, Georgia
St. Benedict's Episcopal School
Interdisciplinary STEAM Specialist
Smyrna, Georgia
St. Benedict's Episcopal School
Arizona School Data Analyst - (AZVA)
Arizona, United States
K12 Inc.
Software Engineer
Portland, OR, US
Northwest Evaluation Association

Read Next

School & District Management Opinion New Resource Tracks School System Reopening
The Return to Learn Tracker identifies the current instructional model of all regular public school districts with three or more schools.
5 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
School & District Management San Francisco School Board Pauses Renaming 44 Schools, Promises to Consult Historians
The renaming of 44 schools in the San Francisco Unified School District is apparently being put on hold after intense blowback.
Greg Keraghosian
1 min read
A pedestrian walks below a sign for Dianne Feinstein Elementary School in San Francisco, on Dec. 17, 2020. The San Francisco Unified School District put the renaming of 44 schools, including Dianne Feinstein Elementary School, on hold after local and national blowback.
A pedestrian walks below a sign for Dianne Feinstein Elementary School in San Francisco, on Dec. 17, 2020. The San Francisco Unified School District put the renaming of 44 schools, including Dianne Feinstein Elementary School, on hold after local and national blowback.<br/><br/>
Jeff Chiu/AP
School & District Management Superintendent Who Led During COVID-19 School Shutdowns Gets Top Honors
Michelle Reid of Washington state's Northshore district, one of the very first to close schools last March, was named National Superintendent of the Year.
3 min read
Michelle Reid, superintendent of the Northshore district in Washington
Michelle Reid, the superintendent of the Northshore district in Washington, was named National Superintendent of the Year.
courtesy of AASA, the School Superintendents Association
School & District Management Is Lunchtime the 'Weak Link' in School Reopening Plans?
It's risky when students are inside and unmasked, experts say. Here are five ways to mitigate that risk and make in-school meals safer.
11 min read
Elementary students in Brownsville, Texas, eat a socially distanced lunch in the school cafeteria. Experts say there are ways to mitigate the risk of contracting COVID-19 even when kids take their masks off to eat.
Elementary students in Brownsville, Texas, eat a socially distanced lunch in the school cafeteria. Experts say there are ways to mitigate the risk of contracting COVID-19 when kids take their masks off to eat.
Denise Cathey/The Brownsville Herald via AP