Opinion
School & District Management Opinion

Experience With Practical Evaluation (III): Today

September 23, 2007 2 min read
  • Save to favorites
  • Print

Today, most k-12 program evaluation is, at best. a single snapshot of what worked - or not - where and when. At worst it’s a political weapon employed in the charter, voucher, privatization, math or reading wars. Typically it’s a less than completely informative marketing tool. Rarely is it employed for the kind practical evaluation described earlier in this series of blog posts.As it happens, post-NAS, RAND’s high profile k-12 evaluation work, much of which has been led by Brian Gill (whose move to Mathematica prompted my thinking about practical evaluation), has offered examples of each. In only one case have I seen any real evidence of the programmatic course corrections “en route” that typified the RAND experience with NAS .

That case is the Pittsburgh study noted last week.

• Edison’s decision to hire RAND in 2000 was surely in part an effort to bolster the firms business credibility and educational legitimacy at a time when both were in doubt. The study itself showed Edison offered school districts roughly the same odds of turning around failing schools as the district itself, at a marginally higher cost. I have seen nothing to suggest how or if Edison used the RAND study to improve its school management services. In the absence of any serious explanation by the firm, I succumb to the temptation of reminding readers of Korean automotive manufacturer Hyndai’s new ad: “Are car companies committed to quality, or to the phrase ‘committed to quality.’”

• RAND’s work on Philadelphia school turn around efforts - also showing no huge difference between schools managed by outsider and those reorganized by the districts. That report became fodder for a methodology debate between RAND and Paul Peterson that simply obscured the bottom line - school management is no silver bullet, it’s just another tool we’re still honing. Whatever role the study may have played in the City of Brotherly Love’s decisions on school management contract, it surely was not the centerpiece of a dispassionate discussion among stakeholders on how to improve their joint enterprise.

Evaluation is a scorecard. Buyers need some means of sorting through their choices, and imperfect means are better than none. But it’s not merely a scorecard, and it’s not just the program that’s being scored - it’s also all the other stakeholders whose efforts make for success or failure. The “scorecard function is not even the most important. Practical evaluation is a means of continuous improvement for the entire system where the school improvement program is introduced. It’s basic purpose is to provide the fact base for ongoing conversations between buyers and sellers and every other group engaged in implementation.

I’m not singling out RAND or Gill for special treatment. They are just examples of a widespread unhelpful social phenomena in the use of evaluation - one the (not so) new (but slow to get started) Society for Research on Education Effectiveness should address as its first priority (but doesn’t).

For more on the related subject of adaptive management.

Related Tags:

The opinions expressed in edbizbuzz are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.

Events

Student Well-Being Webinar After-School Learning Top Priority: Academics or Fun?
Join our expert panel to discuss how after-school programs and schools can work together to help students recover from pandemic-related learning loss.
Budget & Finance Webinar Leverage New Funding Sources with Data-Informed Practices
Address the whole child using data-informed practices, gain valuable insights, and learn strategies that can benefit your district.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Classroom Technology Webinar
ChatGPT & Education: 8 Ways AI Improves Student Outcomes
Revolutionize student success! Don't miss our expert-led webinar demonstrating practical ways AI tools will elevate learning experiences.
Content provided by Inzata

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management Data Data: How Schools Respond to Student Hunger Over the Summer
The end of pandemic-era flexibility for schools and community organizations has translated into fewer students receiving free summer meals.
1 min read
Children enjoy lunches provided by the Brownsville Independent School District on June 8, 2016, at the Olivera Park gymnasium in Brownsville, Texas. The local school district provides free lunches to any child under 18 who needs a meal, regardless of their status as a student with the school district.
Children enjoy lunches provided by the Brownsville Independent School District on June 8, 2016, at the Olivera Park gymnasium in Brownsville, Texas. School districts and other organizations can sign up as summer meal sites to continue providing meals to students once school is out of session.
Jason Hoekema/The Brownsville Herald via AP
School & District Management Online Training Program to Boost Number of Principals of Color Expands
A New York City education college is the latest to join an online principal training program for educators of color and equity-minded leaders.
4 min read
Business like setting, with Black man on a laptop in a corporate conference room or office collaborating with a Black woman
E+/Getty
School & District Management How Can You Tell What Students Need to Succeed at School? Ask Them
Some administrators let students drive purchasing decisions, shape dress code policies, and voice their concerns directly.
4 min read
051223 Lead Sym Mark L jb BS
Chris Ferenzi for Education Week
School & District Management Fewer Students Are Getting Free Summer Meals After Pandemic Waivers End
Summer meal programs are expected to serve fewer students following last summer's end of a federal waiver.
5 min read
Kids line up for lunch outside the Michigan City Area Schools' converted school bus at Weatherstone Village on U.S. 20 in Michigan City, Ind., on July 22, 2021. The bus makes four stops every weekday as part of the Summer Food Program.
Kids line up for lunch outside the Michigan City Area Schools' converted school bus at Weatherstone Village on U.S. 20 in Michigan City, Ind., on July 22, 2021. The bus makes four stops every weekday as part of the Summer Food Program. Summer meal programs are expected to serve fewer students this summer after the expiration of a pandemic-era federal waiver.
Jeff Mayes/The News Dispatch via AP