Opinion
Education Opinion

Experience With Practical Evaluation (II): NAS, RAND and Districts

By Marc Dean Millot — September 22, 2007 2 min read
  • Save to favorites
  • Print

RAND served as an “independent third-party evaluator” throughout the history of New American Schools’ efforts to support the development and dissemination of Comprehensive School Reform models or “designs.” RAND’s evaluation activities were not confined to whether a design “worked” and should continue to be funded - although that was one objective of the work. The nonprofit think tank took a disinterested view of how NAS managed the effort. RAND briefings were also a cornerstone of the several meetings NAS staff and Design Team managers held each year. While the discussion almost always involved whether RAND had its facts right, most of the discussion concerned why the facts were as they were (with caveats), what had happened since RAND collected those facts, and what more needed to be done by teams and NAS to make things better or keep them heading in a positive direction.

As NAS and the Teams shifted from design development to what I called the challenge of “quality at scale” in my first presentation to the NAS board as Director of Design Team Development in 1997, RAND’s evaluation engaged the school districts spawning a new marketplace for school improvement. Up to this point, Designs were certainly not in some end state categorized as “developed”; like all professional services, they were in continuous development. But they were more or less understood as a coherent set of interventions to be implemented in some sequence against a more or less defined criteria. The real challenge was that to date they were in place in a relatively small number of sites, free of charge, with the focused assistance of the designs developers. Now the designs would be implemented in a large number of schools, all over the nation, for a price, with newer staff as well as experienced developers.

The meetings of NAS and Teams were now opened up to the district managers with direct responsibility for implementing the MOUs described yesterday. Once again, RAND’s work not only helped NAS determine which jurisdictions and which teams would continue in the dissemination effort, it gave NAS, districts and teams an unusual opportunity to improve the quality of implementation “in process.” Yes, there was a certain amount of debate over RAND’s facts when NAS, the teams and now district leaders met,; and there was some finger pointing, and examples of “the blame game.” Still, the players recognized their joint responsibility for the effort’s outcomes, their shared interest in its success, and the relationship between their individual work the broader context of district reform.

Whether and where the NAS dissemination efforts led to success is the subject of some debate. My view is that they worked where the school district wanted them to work, and actively supported their implementation. But regardless of this debate, I am quite sure that the model of “practical evaluation” developed in that strategy was essential to whatever level of success was attained, and something any school district attempting systemic reform avoids at its peril.

Next: Practical Evaluation Today.

Related Tags:
Opinion

The opinions expressed in edbizbuzz are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Assessment Webinar
Reflections on Evidence-Based Grading Practices: What We Learned for Next Year
Get real insights on evidence-based grading from K-12 leaders.
Content provided by Otus
Mathematics Webinar How to Build Students’ Confidence in Math
Learn practical tips to build confident mathematicians in our webinar.
Student Achievement K-12 Essentials Forum How to Build and Scale Effective K-12 State & District Tutoring Programs
Join this free virtual summit to learn from education leaders, policymakers, and industry experts on the topic of high-impact tutoring.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Education Briefly Stated: April 16, 2025
Here's a look at some recent Education Week articles you may have missed.
9 min read
Education Quiz ICYMI: Do You Know What 'High-Quality Curriculum' Really Means?
Test your knowledge on the latest news and trends in education.
1 min read
Image of curricula.
iStock/Getty
Education Quiz ICYMI: Lawsuits Over Trump's Education Policies And More
Test your knowledge on the latest news and trends in education.
1 min read
Image of money symbol, books, gavel, and scale of justice.
DigitalVision Vectors
Education Quiz ICYMI: Trump Moves to Shift Special Ed Oversight And More
Test your knowledge on the latest news and trends in education.
1 min read
President Donald Trump signs an executive order on TikTok in the Oval Office of the White House, Monday, Jan. 20, 2025, in Washington.
President Donald Trump signs an executive order on TikTok in the Oval Office of the White House, Monday, Jan. 20, 2025, in Washington.
Evan Vucci/AP