Assessment Opinion

Test Scores Don’t Tell the Whole Story (Q&A)

September 05, 2017 | Corrected: September 06, 2017 6 min read
  • Save to favorites
  • Print

Corrected: A previous version of this essay misstated the date of Schneider’s 2013 Commentary and the school rating Somerville received. It also erroneously suggested he had more than one child.

Jack Schneider, an assistant professor of education at the College of the Holy Cross in Worchester, Mass., (and a former high school teacher and edweek.org blogger) had long studied research around school quality. When he was ready to send his own child to class in Somerville, Mass., and discovered the district’s low rating in an online tool created by The Boston Globe, he called out what he believed was a narrow methodology. How could you classify schools as good or bad based on SAT scores and teacher-student ratios alone? The creator of the tool responded: Think you can do better? Schneider accepted the challenge.

For decades, testing outcomes have been a significant factor in the evaluation of a school’s success. That results, as Schneider wrote in a 2013 Education Week Commentary, in a small number of winners and a large number of losers, and has led to poor reputations for many schools. Such stark divisions increase economic segregation, he said, as schools with strong scores push up property values, price out the working class, and leave low-income children isolated. While the Every Student Succeeds Act requires states to incorporate nontraditional indicators into their accountability systems in an effort to rethink school success, Schneider is urging a shift in mindset at every level of education.

Test Scores Don’t Tell the Whole Story School success is often measured by test scores, but they don’t paint an accurate picture, says Jack Schneider

In his new book, Beyond Test Scores: A Better Way to Measure School Quality, released in August by Harvard University Press, Schneider argues that more comprehensive data can undo harmful accountability systems, help parents and teachers make better decisions for students, and help school leaders more accurately recognize which schools need the most assistance—all of which will improve the way we educate students. Schneider has seen improvements on the ground after creating a school quality framework for his own district that uses multiple measures of student engagement, achievement, and school culture.

The Massachusetts Consortium for Innovative Education Assessment, which Schneider co-founded in 2016, is now working at redefining accountability on a larger scale—in six school districts in his home state and counting.

Commentary Associate Kate Stoltzfus spoke to Schneider recently by phone about how other districts can find expansive ways to define success, thereby creating classrooms that put students—not test scores—first.

Standardized-test scores have been used to measure K-12 public schools’ quality for more than a half century, but accountability as measured by testing has grown increasingly important in recent decades. Why do you think this testing mindset has become so ingrained in U.S. schools?

The numbers produced by standardized tests fit the kind of seemingly objective, quantifiable metrics that have triumphed across American society. By now, many generations of people have come of age with testing simply being a part of life. ESSA and NCLB are merely the latest of these kinds of policy structures. You can go back to the middle of the 20th century and find policymakers describing how dependent they are on test scores for some kind of information about what is going on in local schools. The argument traditionally made for the past 15 years against standardized-test scores is that they are inaccurate and end up being harmful, but there’s something missing from that: We have a better way to do this.

After finding out that your child’s school district was ranked only by SAT scores and student-teacher ratios, you set out to create a new set of tools for measuring school quality. Three years into this work, what changes have you seen at the classroom and district levels?

Educators tend to trust data more when they see that those data align with all the things they are trying to do in the classroom."

We engaged stakeholders in the community in six districts [across Massachusetts]. We talked to students, families, teachers, principals, district administrators—several hundred of them, in fact, to find out what it is they value and what they think good schools do. And then we organized all this into a school quality framework with five major categories: teachers and the teaching environment; school culture; resources; academic learning; and citizenship and well-being. We’ve found that public perceptions of the public school actually improved when people had a wide array of performance data rather than just standardized-test scores, which tend to correlate strongly with family income. We’ve also found that educators tend to trust data more when they see that those data align with all the things they are trying to do in the classroom.

How can more expansive ways of measuring schools be helpful to educators specifically?

They won’t be put in this position where they are being asked to do a job that is different from the job they believe in. Even just to shift from standardized testing to performance assessment (a broader and more comprehensive way of measuring student learning), what we see is teachers feel more empowered. They feel like they have a bit more autonomy to act as professionals.

Some of the data we’re producing are enabling teachers to do some experiments in their classroom, where they want to figure out how to engage students better, how to build stronger relationships with students and [greater] trust in the classroom. These are ultimately the kinds of data points that improve instruction and give educators information they didn’t already have, as opposed to standardized-test reports that will tell educators students are performing below a level of proficiency. An educator who spent 180 days in the classroom with that child probably knew that already.

Test Scores Don’t Tell the Whole Story School success is often measured by test scores, but they don’t paint an accurate picture, says Jack Schneider.

It’s both about the spirit of a school and helping to restore the full purpose of education, and it’s about actionable information. For students and parents and citizens who are interested in public education, it’s about getting better at all of the things that we want to get better at, not just in this one narrow area—the acquisition of the academic content in math and English—that often produces unintended consequences like undermining the rest of the curriculum.

You argue that testing results are often more indicative of students’ socioeconomic status than a school’s quality, and that a testing culture creates stigmas harmful to student success—particularly for low-income students. How so?

Students who start with all kinds of early advantages—not just material advantages, but the academic advantage of having two parents who are college-educated and are emphasizing education in the household and setting a tone and example—those carry into school. And students who have those parents will do better on standardized tests than their peers who don’t grow up in such households. When we look at proficiency scores, we are often seeing the effect not of the school, but of the household on that individual child. Quality-conscious parents with the resources to move, to choose a “good” school or school district, begin to move to particular districts and choose particular schools and bring with them their high-scoring students. And the opposite is true when the school is branded as low-performing. One way to break this cycle is to begin to measure things that do not align so closely with demographic variables.

You say that you want to create better ways to measure schools, not just to better evaluate them individually, but to rethink as a nation how we conceptualize what a “good” school is. How has your definition of a good school been altered?

Most of us in our hearts know what a good school is and does, but we don’t have the language ready and we haven’t conceptualized it as a broader community. The way I tend to think about school quality these days is shaped by the future. Who are the kids I want to meet 10 years from now? That is not a world that is going to be made by drill-and-kill instruction in math and English. It’s a world that is going to be shaped by children discovering their interests and passions and talents in the classroom. It’s shaped by children going to school with people who are not always exactly like them and learning how to work together and who are given opportunities to play and create together. We are limited right now by our policies, tools, and language, but those things can change. We are not limited by our vision or by our shared sense of community.

The interview has been edited for length and clarity.

Related Tags:

Follow the Education Week Commentary section on Facebook and Twitter.
A version of this article appeared in the September 06, 2017 edition of Education Week as What Makes a School Good? It’s More Than Test Scores


English-Language Learners Webinar Helping English-Learners Through Improved Parent Outreach: Strategies That Work
Communicating with families is key to helping students thrive – and that’s become even more apparent during a pandemic that’s upended student well-being and forced constant logistical changes in schools. Educators should pay particular attention
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Mathematics Webinar
Addressing Unfinished Learning in Math: Providing Tutoring at Scale
Most states as well as the federal government have landed on tutoring as a key strategy to address unfinished learning from the pandemic. Take math, for example. Studies have found that students lost more ground
Content provided by Yup Math Tutoring
Classroom Technology Webinar Building Better Blended Learning in K-12 Schools
The pandemic and the increasing use of technology in K-12 education it prompted has added renewed energy to the blended learning movement as most students are now learning in school buildings (and will likely continue

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment State Test Results Are In. Are They Useless?
While states, districts, and schools pore over data from spring 2021 tests, experts urge caution over how to interpret and use the results.
9 min read
FILE - In this Jan. 17, 2016 file photo, a sign is seen at the entrance to a hall for a college test preparation class in Bethesda, Md. The $380 million test coaching industry is facing competition from free or low-cost alternatives in what their founders hope will make the process of applying to college more equitable. Such innovations are also raising questions about the relevance and the fairness of relying on standardized tests in admissions process.
A sign is posted at the entrance to a hall for a test-preparation class. Assessment experts say educators should use data from spring 2021 tests with caution.
Alex Brandon/AP
Assessment Data Young Adolescents' Scores Trended to Historic Lows on National Tests. And That's Before COVID Hit
The past decade saw unprecedented declines in the National Assessment of Educational Progress's longitudinal study.
3 min read
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Assessment Whitepaper
Proven Techniques for Assessing Students with Technology
Dr. Doug Fisher’s proven assessment techniques help your students become active learners and increase their chances for higher learning g...
Content provided by Achieve3000
Assessment Long a Testing Bastion, Florida Plans to End 'Outdated' Year-End Exams
Florida Gov. Ron DeSantis said the state will shift to "progress monitoring" starting in the 2022-23 school year.
5 min read
Florida Governor Ron DeSantis speaks at the opening of a monoclonal antibody site in Pembroke Pines, Fla., on Aug. 18, 2021.
Florida Gov. Ron DeSantis said he believes a new testing regimen is needed to replace the Florida Standards Assessment, which has been given since 2015.
Marta Lavandier/AP