Assessment Commentary

Multiple Measures?

By S. Paul Reville — November 14, 2001 3 min read
The problem with multiple measures is that reasonable people and, in fact, many assessment experts disagree on the basic definition of what they are.

As high stakes become a threat and, finally, a reality for many students in this country, the critics’ outcry about the evils of testing and the inequity of stakes intensifies. Some would like to abolish the entire standards-based reform strategy and revert to the good old days when all standards, if they existed, were local. Some don’t like any form of testing. Others find the tests acceptable, but are in favor of abolishing the stakes while retaining the test for diagnostic purposes. Some would support standards and stakes but only if the assessment system was broadened to include “multiple measures.” Proponents of “multiple measures” generally want to see additional indicators of learning included in the determination of a student’s competency.

The problem is that reasonable people and, in fact, many assessment experts disagree on the basic definition of a “multiple measure.” I have challenged groups of lay people, education advocates, graduate education faculty, assessment gurus, and others to suggest what the other indicators and assessment tools used might be. The answers have been widely divergent and often fuzzy. There is no consensus on the subject of multiple measures, yet the term has become almost a mantra in the discussion about how to improve state assessments.

As a response to the current discord on assessment and stakes, the idea of multiple measures has appeal. It is positive and constructive, building on the principles embodied in the strategy of holding all students to a high standard, measuring progress, and making performance count. Policymakers are intrigued with the notion that something could be added to the assessment process to make it fuller and fairer, and thereby silence some of the shrillest critics. But policymakers can’t get very far with the multiple-measures concept until proponents become clearer about what it actually means.

Policymakers can't get very far with the multiple-measures concept until proponents become clearer about what it actually means.

In order to join this conversation in Massachusetts, the Massachusetts Reform Review Commission convened a workshop for various stakeholders who wanted to explore the multiple-measures concept. Researchers, policymakers, advocates, and practitioners came together to tackle the elusive question of “what assessment tools or indicators would you add to the current MCAS “the Massachusetts Comprehensive Assessment System—"to make it fairer and more comprehensive?”

As the chairman of the commission, I challenged the group to devise a set of proposed “multiple measures” that met six criteria that I felt were essential for viability. These included the following:

  • Validity. Do the additional tools accurately measure learning embodied in the standards?
  • Reliability. Will the additional measure repeatedly and accurately generate consistent results in a variety of circumstances?
  • Transparency. Would its use be understandable and clear to the public, parents, and educators?
  • Practicality. Is it relatively easy to compile or administer? Is it feasible for teachers and students?
  • Affordability. Are the costs reasonable and affordable?
  • Political Feasibility. Could any self-respecting politician stand up and support the use of this measure in public?

All of these criteria are challenging to meet. Many of them are matters of degree and subject to human judgment. For instance, how valid and reliable does an instrument or data set need to be? Despite the obvious difficulty of arriving at such determinations, I am confident that appropriate standards could be agreed in each of the areas.

In Massachusetts, we are challenging educators, policymakers, researchers, and especially critics of the current assessment system to propose some additional performance measures, some “multiple measures” that meet the criteria I have outlined. The commission’s first workshop yielded a promising beginning to a process for developing some additional measures of the kind contemplated in the state’s education reform act of 1993. We intend to invest further resources in continuing this work.

But the workshop conversations also made it apparent that, as we suspected, “multiple measures” are far easier said than done. Both locally and nationally, much more work needs to be done in articulating the principles and practice of comprehensive, effective, and useful state assessment indicators and tools.

S. Paul Reville chairs the Massachusetts Education Reform Review Commission and is the executive director of the Pew Forum on Standards Based Reform. The forum is based at Harvard University’s graduate school of education in Cambridge, Mass.

A version of this article appeared in the November 14, 2001 edition of Education Week as Multiple Measures?


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Classroom Technology Webinar
Educator-Driven EdTech Design: Help Shape the Future of Classroom Technology
Join us for a collaborative workshop where you will get a live demo of GoGuardian Teacher, including seamless new integrations with Google Classroom, and participate in an interactive design exercise building a feature based on
Content provided by GoGuardian
School & District Management Live Online Discussion A Seat at the Table With Education Week: What Did We Learn About Schooling Models This Year?
After a year of living with the pandemic, what schooling models might we turn to as we look ahead to improve the student learning experience? Could year-round schooling be one of them? What about online
School & District Management Webinar What's Ahead for Hybrid Learning: Putting Best Practices in Motion
It’s safe to say hybrid learning—a mix of in-person and remote instruction that evolved quickly during the pandemic—is probably here to stay in K-12 education to some extent. That is the case even though increasing

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment Opinion AP Exams Can't Be Business as Usual This Year
The College Board seems unconcerned with the collateral damage of its pandemic approach, writes an assistant superintendent of curriculum and instruction.
Pete Bavis
5 min read
Illustration of large boat in turbulent waters with other smaller boats falling into the abyss.
iStock/Getty Images Plus
Assessment Federal Lawmakers Urge Miguel Cardona to Let States Cancel Tests, Highlighting Discord
A letter from Democratic members to the new education secretary calls for an end to the "flawed" system of annual standardized exams.
3 min read
Jamaal Bowman speaks to reporters after voting at a polling station inside Yonkers Middle/High School in Yonkers, N.Y. on June 23, 2020.
Jamaal Bowman speaks to reporters after voting at a polling station inside Yonkers Middle/High School in Yonkers, N.Y. on June 23, 2020.
John Minchillo/AP
Assessment How Two Years of Pandemic Disruption Could Shake Up the Debate Over Standardized Testing
Moves to opt out of state tests and change how they're given threaten to reignite fights over high-stakes assessments.
9 min read
Image of a student at a desk.
Assessment A Plan for Standardized Test Scores During the Pandemic Has Gotten States' Attention
A testing expert says his idea would provide helpful data with key context, but said other measures about student well-being are crucial.
7 min read
HS class 1257213326