Opinion
School & District Management Opinion

“Take Aways” from Two Months of What Works Clearinghouse Reviews

September 15, 2007 5 min read

For the most part public schools do not develop their own curriculum or pedagogy - they buy them, primarily from private enterprise, and mostly from for-profits. I find it ironic that: 1) outsourcing the most central of public education’s core teaching and learning functions is never part of the “privatization” debate; and 2) the quality of those products services and programs didn’t become a subject of serious interest until No Child Left Behind. Yet “god forbid” we allow private enterprise to operate public schools under charter or contract, or manage teachers’ professional development. Given the discrepancy, it’s hard to believe that debate has much to do with what kids need rather than what adults want.

Speaking of quality, and the Reading First fiasco notwithstanding, program efficacy is supposed to matter today. Educators, indeed taxpayers, have a desperate need for independent reviews of program efficacy. In their absence, marketing dominates the sales race.
After a very slow start, the What Works Clearinghouse (WWC), a Department of Education Institute for Education Sciences’ activity (contracted out to a consortium of evaluation organizations) to examine the evidence providers use to claim their programs are grounded in “Scientifically Based Research” (SBR), is starting to pump out reviews across a range of subjects, grades and providers.

The sixteen reviews covered below happens to coincide with September’s two-month catch-up issue of School Improvement Industry Research and Evaluation Announcements http://www.k-12network.com(we vacation in August). There’s something to be learned from the list:

August 13, 2007:

Earobics® - The WWC found Earobics® to have positive effects on alphabetics and no discernible effects on fluency.

Ladders to Literacy for Kindergarten Student… .potentially positive effects on alphabetics and fluency and mixed effects on comprehension.

Success for All…. potentially positive effects on alphabetics and general reading achievement and mixed effects on comprehension.

Voyager Universal Literacy System…. potentially positive effects on alphabetics and potentially negative effects on comprehension.

July 30, 2007:

Doors to Discovery…. no discernible effects on oral language, print knowledge, and phonological processing.

Let’s Begin with the Letter People®…. no discernible effects on oral language and potentially positive effects on print knowledge and phonological processing.

Pre-K Mathematics - The WWC found Pre-K Mathematics combined with DLM Early Childhood Express software to have positive effects on mathematics achievement.

Project GRAD…. no discernible effects on progressing in school or on completing school.

Waterford Early Reading Level One…. no discernible effects on oral language or on print knowledge.

July 23, 2007:

SRA Real Math Building Blocks PreK…. positive effects on mathematics achievement.

July 16, 2007:

Cooperative Integrated Reading and Composition® (CIRC)…. no discernible effects for comprehension.

Peer-Assisted Learning Strategies (PALS)©…. potentially positive effects on alphabetics, fluency, and comprehension.

Beginning Reading: Read Naturally….- no discernible effects on fluency and reading comprehension.

Beginning Reading: Waterford Early Reading Program…. potentially positive effects on alphabetics and no discernible effects on comprehension.

Talent Development High Schools…. no discernible effects on progressing in school or completing school.

Early Childhood Education: Literacy Express…. positive effects on print knowledge and phonological processing, potentially positive effects on oral language and math, and no discernible effects on cognition.

Observations

1. If we take this list and divide it into a crude buyers guide figure according to programs that have some positive evidence and those that don’t, and providers who are nonprofit or for profit, here’s what we get:

Some Positive Finding

For-Profit
• Earobics
• Voyager Literacy
• Pre-K Mathematics
• Waterford Early Reading Program™

Nonprofit
• Success for All
• Ladders to Literacy
• PALS
• Literacy Express

No Positive Finding

For-Profit
• Doors to Discovery
• Let’s Begin with the Letter People
• SRA Real Math
• Read Naturally
• Waterford Early Reading Level One

Nonprofit
• Cooperative Integrated Reading and Composition
• Project GRAD
• Talent Development High Schools

There’s no great quality advantage tied to the label “nonprofit” or any great black mark to “for profit.” Both groups have programs with and without evidence of efficacy. The same for-profit provider can have programs with and without evidence – take Waterford, So can the same nonproit - Cooperative Integrated Reading and Composition is offered by the Success for All Foundation..

2. Our measures of program efficacy are crude and imperfect, but they are still a far better guide than nothing - or marketing brochures. The question now is whether Congress should allow states to use federal funds to buy programs that can’t jump this bar, when there does seem to be a reasonable range of choice. In short, if you can’t pass WWC review with at least some positive finding, can your program really be said to rest on scientifically based research, and should taxpayers be subsidizing a school’s data-free decision to buy it?

3. There’s some kind of cost-benefit analysis that needs to be done by educators before any for any purchase. Still, at any given price what matters to kids is outcomes, not whether the provider has a cash surplus after paying out expenses to deliver the program, or whether that surplus is retained by a nonprofit or passed along to a company’s owners.

4. Jim Ketelsen of Project GRAD and Jim McPartland of Talent Development were no less celebrated a few years ago than John Schnur of New Leaders for New Schools is today, and given the unfairness of scheduling could just as easily taken his place at the recent Gang of Forty-Four hearings on NCLB. And, as capable a manager as John is (and I admire him greatly), he is even less able to muster enough evidence of the efficacy of New Leaders for New Schools to meet WWC’s low bar. If Congress is really serious about the “supply side” of school reform, it has to turn its focus away from philanthropy and its never-ending array of grant-driven pilots unsullied by evaluation. It has to get serious about SBR and about the fee-driven for- and nonprofit providers offering the vast bulk of products and services aimed school improvement who are in this business for the long haul – and the vast amount of potential private sector capital that isn’t flowing to them. Congress has got to start thinking about how to stimulate this industry around “what works”, and only what works..

Related Tags:

The opinions expressed in edbizbuzz are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Teaching Webinar
Meeting the Moment: Accelerating Equitable Recovery and Transformative Change
Educators are deciding how best to re-establish routines such as everyday attendance, rebuild the relationships for resilient school communities, and center teaching and learning to consciously prioritize protecting the health and overall well-being of students
Content provided by Campaign for Grade-Level Reading
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Reading & Literacy Webinar
Addressing Learning Loss: What Schools Need to Accelerate Reading Instruction in K-3
When K-3 students return to classrooms this fall, there will be huge gaps in foundational reading skills. Does your school or district need a plan to address learning loss and accelerate student growth? In this
Content provided by PDX Reading
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Equity & Diversity Webinar
Culturally Relevant Pedagogy to Advance Educational Equity
Schools are welcoming students back into buildings for full-time in-person instruction in a few short weeks and now is the perfect time to take a hard look at both our practices and systems to build
Content provided by PowerMyLearning

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management Opinion COVID-19 Ripped Through Our Emotional Safety Net. Here’s How My District Responded
Three years after overhauling its approach to student mental health, one California district found itself facing a new crisis.
Jonathan Cooper
2 min read
A young man stands under a street light on a lonely road.
Vanessa Solis/Education Week and Getty Images
School & District Management Opinion Students Need Better Connections. To Wi-Fi, Yes, But Also to Teachers
We have to fix our digital divide, but let’s not lose sight of the relationship divide, writes one superintendent.
Susan Enfield
2 min read
A teacher checks in on a remote student.
Vanessa Solis/Education Week and Getty Images
School & District Management Opinion Superintendents Have Weathered a Lot of Vitriol This Year. What Have We Learned?
The pandemic turned district leaders into pioneers, writes one superintendent. We had to band together to make it through.
Matthew Montgomery
2 min read
A person walks from a vast empty space towards a team of people.
Vanessa Solis/Education Week and Getty Images
School & District Management Opinion Critics Complain My District Doesn’t Really Need Relief Aid. If They Only Knew…
District expenditures have ballooned in the pandemic, but many critics expect the opposite. How can leaders set the record straight?
Theresa Rouse
2 min read
A business person convinces colleagues by presenting a plan.
Vanessa Solis/Education Week and Getty Images