Find your next job fast at the Jan. 28 Virtual Career Fair. Register now.
Education Opinion

‘Defies Measurement’ Illustrates Failures of Test-Focused Policy

By David B. Cohen — June 03, 2015 4 min read

In “Defies Measurement,” teacher-turned-filmmaker Shannon Puckett gathers the recollections and reflections of twenty-three former students, parents, and teachers from Chipman Middle School in Alameda, California, and illustrates how a nurturing school community was gradually dismantled by the test-and-punish dynamics of education reform under No Child Left Behind. Puckett, who taught at Chipman and quit because of the chages following from NCLB, also contextualizes the eventual closure of the school, and the devaluation of what it stood for, in the broader context of education reform and accountability efforts nationwide. To that end, an impressive lineup of over twenty policy experts and public education activists also make appearances in the film. It was produced on a limited budget, and received over $26,000 from Kickstarter backers (including me). From a production standpoint, “Defies Measurement” is not quite as slick as some education documentaries out there in terms of its videography, audio, and editing, but it has wonderful archival material from the school, fine graphics, and most importantly, excellent content from a variety of useful perspectives.

Prior to NCLB, Chipman Middle School had created an environment that nurtured children and promoted experiential learning. Outdoor education, participation in the arts, and other creative activities illustrated in the film made the school a place that children wanted to go, to be part of. By promoting a safe, community atmosphere, teachers and school leaders made Chipman a place where students could also take personal and intellectual risks. Former students interviewed in the film, now in adulthood, cite their time at the school as essential to their development as learners and as people. A former teacher points out that it wasn’t complicated or expensive, but simply involved dedicated and caring teachers doing what they knew would help children learn and grow.

The increased importance of standardized test scores posed an existential threat to Chipman, though it’s clear in the film and from viewing the broader results of NCLB that being labeled a “failure” would be the eventual fate of almost every public school - especially those serving students in poverty, learning English, or with special educational needs. As the pressure mounted to raise test scores, Chipman teachers embraced new methods of teaching, even earning the attention and praise of the Bush administration; Laura Bush visited the school and commended their work. Their prior model and structure gradually fell apart, students enjoyed the school less, test scores failed to meet targets, and in 2010, Chipman was closed and replaced by a charter school.

Former Chipman teachers in the film, looking back on that time period, express serious personal regrets, though it’s not clear that alternative choices would have saved their school either. One (slightly edited) conversation that stood out to me involves former teachers Sylvia Kahn and Tracy Allegrotti (whose daughter also contributes a useful perspective as a former student).

Kahn: It shouldn’t have been either/or. We should have been able to give the kids that reading instruction, and give everybody this phenomenally rich, creative environment.

Allegrotti: Don’t you think, on some level... we all sold out, that we knew - and I really feel like I own this - that we knew what was right for kids? And we were told, “Now you have to do this,” and ... I went, “Okay, you’re telling me what to do.”

Kahn: I have great remorse about decisions that were made. I don’t feel that it was selling out. I feel I was duped. I think maybe I was more trusting of what might be coming down from a larger level, and so, when we started to make the changes at Chipman that started to dismantle anything that was good and beautiful, and taught children how to think and learn and be independent - that they knew what it was we needed to do. I had faith that they knew something that we didn’t know.

The personal stories are effectively balanced by the presence of researchers and policy experts who put the Chipman story into the broader frame of American education. Alan Stotskopf traces the eugenic origins of standardized testing (with legacies that still play out in schools, sometimes in ways educators may not even recognize). Linda Darling-Hammond discusses the limitations of test data to draw conclusions about students and schools, especially when used for international comparisons. David Berliner, Diane Ravitch, David Kirp, Julian Vasquez-Heilig, Howard Gardner, Mark Naison, Tony Wagner, and Alfie Kohn all make appearances to discuss the misuses of testing and test data, the flawed assumptions in reform policies, and the dramatic shift of money and power towards private entities seeing public education as a marketplace. Education bloggers and activists are also well represented, with Anthony Cody, Mercedes Schneider, and Karran Harper-Royal discussing the disempowerment of local schools and communities under current education refrom models.

Make some time to watch “Defies Measurement” - available to view or download from Vimeo.

Images: “Defies Measurement” poster by permission of the filmmaker; data screenshot from CA Dept. of Education.

The opinions expressed in Road Trips in Education are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
School & District Management Webinar
Branding Matters. Learn From the Pros Why and How
Learn directly from the pros why K-12 branding and marketing matters, and how to do it effectively.
Content provided by EdWeek Top School Jobs
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
School & District Management Webinar
How to Make Learning More Interactive From Anywhere
Join experts from Samsung and Boxlight to learn how to make learning more interactive from anywhere.
Content provided by Samsung
Teaching Live Online Discussion A Seat at the Table With Education Week: How Educators Can Respond to a Post-Truth Era
How do educators break through the noise of disinformation to teach lessons grounded in objective truth? Join to find out.

EdWeek Top School Jobs

Elementary Teacher
Madison, Wisconsin
One City Schools
Special Education Teacher
Chicago, Illinois
JCFS Chicago
Elementary Teacher - Scholars Academy
Madison, Wisconsin
One City Schools
Clinical Director
Garden Prairie, IL, US
Camelot Education

Read Next

Education Briefly Stated Briefly Stated: January 13, 2021
Here's a look at some recent Education Week articles you may have missed.
8 min read
Education Obituary In Memory of Michele Molnar, EdWeek Market Brief Writer and Editor
EdWeek Market Brief Associate Editor Michele Molnar, who was instrumental in launching the publication, succumbed to cancer.
5 min read
Education Briefly Stated Briefly Stated: December 9, 2020
Here's a look at some recent Education Week articles you may have missed.
8 min read
Education Briefly Stated Briefly Stated: Stories You May Have Missed
A collection of articles from the previous week that you may have missed.
8 min read