Education Opinion

Experience With Practical Evaluation (II): NAS, RAND and Districts

By Marc Dean Millot — September 22, 2007 2 min read
  • Save to favorites
  • Print

RAND served as an “independent third-party evaluator” throughout the history of New American Schools’ efforts to support the development and dissemination of Comprehensive School Reform models or “designs.” RAND’s evaluation activities were not confined to whether a design “worked” and should continue to be funded - although that was one objective of the work. The nonprofit think tank took a disinterested view of how NAS managed the effort. RAND briefings were also a cornerstone of the several meetings NAS staff and Design Team managers held each year. While the discussion almost always involved whether RAND had its facts right, most of the discussion concerned why the facts were as they were (with caveats), what had happened since RAND collected those facts, and what more needed to be done by teams and NAS to make things better or keep them heading in a positive direction.

As NAS and the Teams shifted from design development to what I called the challenge of “quality at scale” in my first presentation to the NAS board as Director of Design Team Development in 1997, RAND’s evaluation engaged the school districts spawning a new marketplace for school improvement. Up to this point, Designs were certainly not in some end state categorized as “developed”; like all professional services, they were in continuous development. But they were more or less understood as a coherent set of interventions to be implemented in some sequence against a more or less defined criteria. The real challenge was that to date they were in place in a relatively small number of sites, free of charge, with the focused assistance of the designs developers. Now the designs would be implemented in a large number of schools, all over the nation, for a price, with newer staff as well as experienced developers.

The meetings of NAS and Teams were now opened up to the district managers with direct responsibility for implementing the MOUs described yesterday. Once again, RAND’s work not only helped NAS determine which jurisdictions and which teams would continue in the dissemination effort, it gave NAS, districts and teams an unusual opportunity to improve the quality of implementation “in process.” Yes, there was a certain amount of debate over RAND’s facts when NAS, the teams and now district leaders met,; and there was some finger pointing, and examples of “the blame game.” Still, the players recognized their joint responsibility for the effort’s outcomes, their shared interest in its success, and the relationship between their individual work the broader context of district reform.

Whether and where the NAS dissemination efforts led to success is the subject of some debate. My view is that they worked where the school district wanted them to work, and actively supported their implementation. But regardless of this debate, I am quite sure that the model of “practical evaluation” developed in that strategy was essential to whatever level of success was attained, and something any school district attempting systemic reform avoids at its peril.

Next: Practical Evaluation Today.

The opinions expressed in edbizbuzz are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.


Special Education K-12 Essentials Forum Innovative Approaches to Special Education
Join this free virtual event to explore innovations in the evolving landscape of special education.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Curriculum Webinar
STEM Fusion: Empowering K-12 Education through Interdisciplinary Integration
Join our webinar to learn how integrating STEM with other subjects can revolutionize K-12 education & prepare students for the future.
Content provided by Project Lead The Way
School & District Management Webinar How Pensions Work: Why It Matters for K-12 Education
Panelists explain the fundamentals of teacher pension finances — how they are paid for, what drives their costs, and their impact on K-12 education.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Education Briefly Stated: June 12, 2024
Here's a look at some recent Education Week articles you may have missed.
9 min read
Education Briefly Stated: May 29, 2024
Here's a look at some recent Education Week articles you may have missed.
9 min read
Education Briefly Stated: May 8, 2024
Here's a look at some recent Education Week articles you may have missed.
8 min read
Education Briefly Stated: April 17, 2024
Here's a look at some recent Education Week articles you may have missed.
8 min read