Education Opinion

‘Defies Measurement’ Illustrates Failures of Test-Focused Policy

By David B. Cohen — June 03, 2015 4 min read
  • Save to favorites
  • Print

In “Defies Measurement,” teacher-turned-filmmaker Shannon Puckett gathers the recollections and reflections of twenty-three former students, parents, and teachers from Chipman Middle School in Alameda, California, and illustrates how a nurturing school community was gradually dismantled by the test-and-punish dynamics of education reform under No Child Left Behind. Puckett, who taught at Chipman and quit because of the chages following from NCLB, also contextualizes the eventual closure of the school, and the devaluation of what it stood for, in the broader context of education reform and accountability efforts nationwide. To that end, an impressive lineup of over twenty policy experts and public education activists also make appearances in the film. It was produced on a limited budget, and received over $26,000 from Kickstarter backers (including me). From a production standpoint, “Defies Measurement” is not quite as slick as some education documentaries out there in terms of its videography, audio, and editing, but it has wonderful archival material from the school, fine graphics, and most importantly, excellent content from a variety of useful perspectives.

Prior to NCLB, Chipman Middle School had created an environment that nurtured children and promoted experiential learning. Outdoor education, participation in the arts, and other creative activities illustrated in the film made the school a place that children wanted to go, to be part of. By promoting a safe, community atmosphere, teachers and school leaders made Chipman a place where students could also take personal and intellectual risks. Former students interviewed in the film, now in adulthood, cite their time at the school as essential to their development as learners and as people. A former teacher points out that it wasn’t complicated or expensive, but simply involved dedicated and caring teachers doing what they knew would help children learn and grow.

The increased importance of standardized test scores posed an existential threat to Chipman, though it’s clear in the film and from viewing the broader results of NCLB that being labeled a “failure” would be the eventual fate of almost every public school - especially those serving students in poverty, learning English, or with special educational needs. As the pressure mounted to raise test scores, Chipman teachers embraced new methods of teaching, even earning the attention and praise of the Bush administration; Laura Bush visited the school and commended their work. Their prior model and structure gradually fell apart, students enjoyed the school less, test scores failed to meet targets, and in 2010, Chipman was closed and replaced by a charter school.

Former Chipman teachers in the film, looking back on that time period, express serious personal regrets, though it’s not clear that alternative choices would have saved their school either. One (slightly edited) conversation that stood out to me involves former teachers Sylvia Kahn and Tracy Allegrotti (whose daughter also contributes a useful perspective as a former student).

Kahn: It shouldn’t have been either/or. We should have been able to give the kids that reading instruction, and give everybody this phenomenally rich, creative environment.

Allegrotti: Don’t you think, on some level... we all sold out, that we knew - and I really feel like I own this - that we knew what was right for kids? And we were told, “Now you have to do this,” and ... I went, “Okay, you’re telling me what to do.”

Kahn: I have great remorse about decisions that were made. I don’t feel that it was selling out. I feel I was duped. I think maybe I was more trusting of what might be coming down from a larger level, and so, when we started to make the changes at Chipman that started to dismantle anything that was good and beautiful, and taught children how to think and learn and be independent - that they knew what it was we needed to do. I had faith that they knew something that we didn’t know.

The personal stories are effectively balanced by the presence of researchers and policy experts who put the Chipman story into the broader frame of American education. Alan Stotskopf traces the eugenic origins of standardized testing (with legacies that still play out in schools, sometimes in ways educators may not even recognize). Linda Darling-Hammond discusses the limitations of test data to draw conclusions about students and schools, especially when used for international comparisons. David Berliner, Diane Ravitch, David Kirp, Julian Vasquez-Heilig, Howard Gardner, Mark Naison, Tony Wagner, and Alfie Kohn all make appearances to discuss the misuses of testing and test data, the flawed assumptions in reform policies, and the dramatic shift of money and power towards private entities seeing public education as a marketplace. Education bloggers and activists are also well represented, with Anthony Cody, Mercedes Schneider, and Karran Harper-Royal discussing the disempowerment of local schools and communities under current education refrom models.

Make some time to watch “Defies Measurement” - available to view or download from Vimeo.

Images: “Defies Measurement” poster by permission of the filmmaker; data screenshot from CA Dept. of Education.

The opinions expressed in Road Trips in Education are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.

Commenting has been disabled on edweek.org effective Sept. 8. Please visit our FAQ section for more details. To get in touch with us visit our contact page, follow us on social media, or submit a Letter to the Editor.


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Teaching Webinar
6 Key Trends in Teaching and Learning
As we enter the third school year affected by the pandemic—and a return to the classroom for many—we come better prepared, but questions remain. How will the last year impact teaching and learning this school
Content provided by Instructure
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Equity & Diversity Webinar
Leadership for Racial Equity in Schools and Beyond
While the COVID-19 pandemic continues to reveal systemic racial disparities in educational opportunity, there are revelations to which we can and must respond. Through conscientious efforts, using an intentional focus on race, school leaders can
Content provided by Corwin
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Equity & Diversity Webinar
Evaluating Equity to Drive District-Wide Action this School Year
Educational leaders are charged with ensuring all students receive equitable access to a high-quality education. Yet equity is more than an action. It is a lens through which we continuously review instructional practices and student
Content provided by BetterLesson

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Education Schools Get the Brunt of Latest COVID Wave in South Carolina
In the past few weeks, South Carolina has set records for COVID-19 hospitalizations and new cases have approached peak levels of last winter.
4 min read
Two Camden Elementary School students in masks listen as South Carolina Gov. Henry McMaster talks about steps the school is taking to fight COVID-19, Wednesday, Sept. 15, 2021, in Camden, S.C. McMaster has adamantly and repeatedly come out against requiring masks in schools even as the average number of daily COVID-19 cases in the state has risen since early June. (AP Photo/Jeffrey Collins)
Education More States Are Requiring Schools to Teach Native American History and Culture
Advocates say their efforts have gained some momentum with the nation’s reckoning over racial injustice since the killing of George Floyd.
3 min read
A dancer participates in an intertribal dance at Schemitzun on the Mashantucket Pequot Reservation in Mashantucket, Conn., Saturday, Aug. 28, 2021. Connecticut and a handful of other states have recently decided to mandate students be taught about Native American culture and history. (AP Photo/Jessica Hill)
Education Judge's Temporary Order Allows Iowa Schools to Mandate Masks
A federal judge ordered the state to immediately halt enforcement of a law that prevents school boards from ordering masks to be worn.
4 min read
Iowa Gov. Kim Reynolds speaks to reporters following a news conference, Thursday, Aug. 19, 2021, in West Des Moines, Iowa. Reynolds lashed out at President Joe Biden Thursday after he ordered his education secretary to explore possible legal action against states that have blocked school mask mandates and other public health measures meant to protect students against COVID-19. Reynolds, a Republican, has signed a bill into law that prohibits school officials from requiring masks, raising concerns as delta variant virus cases climb across the state and schools resume classes soon. (AP Photo/Charlie Neibergall)
Education Hurricane Ida Deals New Blow to Louisiana Schools Struggling to Reopen
The opening of the school year offered teachers a chance to fully assess the pandemic's effects, only to have students forced out again.
8 min read
Six-year-old Mary-Louise Lacobon sits on a fallen tree beside the remnants of her family's home destroyed by Hurricane Ida, in Dulac, La., on Sept. 4, 2021. Louisiana students, who were back in class after a year and a half of COVID-19 disruptions kept many of them at home, are now missing school again after Hurricane Ida. A quarter-million public school students statewide have no school to report to, though top educators are promising a return is, at most, weeks away, not months.
Six-year-old Mary-Louise Lacobon sits on a fallen tree beside the remnants of her family's home destroyed by Hurricane Ida, in Dulac, La., on Sept. 4, 2021.
John Locher/AP