Opinion
School & District Management Opinion

Experience With Practical Evaluation (III): Today

September 23, 2007 2 min read

Today, most k-12 program evaluation is, at best. a single snapshot of what worked - or not - where and when. At worst it’s a political weapon employed in the charter, voucher, privatization, math or reading wars. Typically it’s a less than completely informative marketing tool. Rarely is it employed for the kind practical evaluation described earlier in this series of blog posts.As it happens, post-NAS, RAND’s high profile k-12 evaluation work, much of which has been led by Brian Gill (whose move to Mathematica prompted my thinking about practical evaluation), has offered examples of each. In only one case have I seen any real evidence of the programmatic course corrections “en route” that typified the RAND experience with NAS .

That case is the Pittsburgh study noted last week.

• Edison’s decision to hire RAND in 2000 was surely in part an effort to bolster the firms business credibility and educational legitimacy at a time when both were in doubt. The study itself showed Edison offered school districts roughly the same odds of turning around failing schools as the district itself, at a marginally higher cost. I have seen nothing to suggest how or if Edison used the RAND study to improve its school management services. In the absence of any serious explanation by the firm, I succumb to the temptation of reminding readers of Korean automotive manufacturer Hyndai’s new ad: “Are car companies committed to quality, or to the phrase ‘committed to quality.’”

• RAND’s work on Philadelphia school turn around efforts - also showing no huge difference between schools managed by outsider and those reorganized by the districts. That report became fodder for a methodology debate between RAND and Paul Peterson that simply obscured the bottom line - school management is no silver bullet, it’s just another tool we’re still honing. Whatever role the study may have played in the City of Brotherly Love’s decisions on school management contract, it surely was not the centerpiece of a dispassionate discussion among stakeholders on how to improve their joint enterprise.

Evaluation is a scorecard. Buyers need some means of sorting through their choices, and imperfect means are better than none. But it’s not merely a scorecard, and it’s not just the program that’s being scored - it’s also all the other stakeholders whose efforts make for success or failure. The “scorecard function is not even the most important. Practical evaluation is a means of continuous improvement for the entire system where the school improvement program is introduced. It’s basic purpose is to provide the fact base for ongoing conversations between buyers and sellers and every other group engaged in implementation.

I’m not singling out RAND or Gill for special treatment. They are just examples of a widespread unhelpful social phenomena in the use of evaluation - one the (not so) new (but slow to get started) Society for Research on Education Effectiveness should address as its first priority (but doesn’t).

For more on the related subject of adaptive management.

Related Tags:

The opinions expressed in edbizbuzz are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.

Events

Classroom Technology Webinar Making Big Technology Decisions: Advice for District Leaders, Principals, and Teachers
Educators at all levels make decisions that can have a huge impact on students. That’s especially true when it comes to the use of technology, which was activated like never before to help students learn
Professional Development Webinar Expand Digital Learning by Expanding Teacher Training
This discussion will examine how things have changed and offer guidance on smart, cost-effective ways to expand digital learning efforts and train teachers to maximize the use of new technologies for learning.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
The Social-Emotional Impact of the Covid-19 Pandemic on American Schoolchildren
Hear new findings from an analysis of our 300 million student survey responses along with district leaders on new trends in student SEL.
Content provided by Panorama

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management Opinion One Simple Way for Principals to Boost Students’ Unfinished Learning
Instruction improves when teachers remain in their current grades, write researchers Heather C. Hill and Susanna Loeb.
5 min read
Conceptual illustration of people running around a career track.
Vanessa Solis/Education Week and Getty Images
School & District Management Opinion How Did Ed. Leaders Decide Whether to Reopen Schools In Person Last Fall?
As school leaders face tough reopening decisions amid Delta variant concerns, we must understand how those decisions were made last year.
Douglas N. Harris & Katharine O. Strunk
4 min read
Protesters rally outside the San Diego Unified School District headquarters demanding that schools reopen for in-person learning and that voters oust some of the sitting school board members on Oct. 27, 2020 in San Diego.
Protesters rally outside the San Diego Unified School District headquarters demanding that schools reopen for in-person learning and that voters oust some of the sitting school board members on Oct. 27, 2020 in San Diego.
Sam Hodgson/The San Diego Union-Tribune via TNS
School & District Management Opinion Schools Faced a Massive Systems Failure During the Pandemic. How Do We Fix It?
Education leaders can (and must) find common purpose in the face of COVID-19, writes one superintendent.
Laura Kagy
2 min read
Hands hold up gears together.
Vanessa Solis/Education Week and Getty Images
School & District Management 4 Ways to Keep Staff and Students Safe From the Delta Variant
Just as schools reopen, a super-contagious COVID-19 variant is infecting people nationwide at alarming rates. Here's what schools can do.
5 min read
Students and parents walk into school on the first day of school at Enrique S. Camarena Elementary School on July 21, 2021, in Chula Vista, Calif.
Students and parents walk into school on the first day of school at Enrique S. Camarena Elementary School on July 21, 2021, in Chula Vista, Calif.
Denis Poroy/AP