Opinion
Assessment Commentary

Test Scores Don’t Tell the Whole Story (Q&A)

September 05, 2017 | Corrected: September 06, 2017 6 min read

Corrected: A previous version of this essay misstated the date of Schneider’s 2013 Commentary and the school rating Somerville received. It also erroneously suggested he had more than one child.

Jack Schneider, an assistant professor of education at the College of the Holy Cross in Worchester, Mass., (and a former high school teacher and edweek.org blogger) had long studied research around school quality. When he was ready to send his own child to class in Somerville, Mass., and discovered the district’s low rating in an online tool created by The Boston Globe, he called out what he believed was a narrow methodology. How could you classify schools as good or bad based on SAT scores and teacher-student ratios alone? The creator of the tool responded: Think you can do better? Schneider accepted the challenge.

For decades, testing outcomes have been a significant factor in the evaluation of a school’s success. That results, as Schneider wrote in a 2013 Education Week Commentary, in a small number of winners and a large number of losers, and has led to poor reputations for many schools. Such stark divisions increase economic segregation, he said, as schools with strong scores push up property values, price out the working class, and leave low-income children isolated. While the Every Student Succeeds Act requires states to incorporate nontraditional indicators into their accountability systems in an effort to rethink school success, Schneider is urging a shift in mindset at every level of education.

Test Scores Don’t Tell the Whole Story School success is often measured by test scores, but they don’t paint an accurate picture, says Jack Schneider

In his new book, Beyond Test Scores: A Better Way to Measure School Quality, released in August by Harvard University Press, Schneider argues that more comprehensive data can undo harmful accountability systems, help parents and teachers make better decisions for students, and help school leaders more accurately recognize which schools need the most assistance—all of which will improve the way we educate students. Schneider has seen improvements on the ground after creating a school quality framework for his own district that uses multiple measures of student engagement, achievement, and school culture.

The Massachusetts Consortium for Innovative Education Assessment, which Schneider co-founded in 2016, is now working at redefining accountability on a larger scale—in six school districts in his home state and counting.

Commentary Associate Kate Stoltzfus spoke to Schneider recently by phone about how other districts can find expansive ways to define success, thereby creating classrooms that put students—not test scores—first.

Standardized-test scores have been used to measure K-12 public schools’ quality for more than a half century, but accountability as measured by testing has grown increasingly important in recent decades. Why do you think this testing mindset has become so ingrained in U.S. schools?

The numbers produced by standardized tests fit the kind of seemingly objective, quantifiable metrics that have triumphed across American society. By now, many generations of people have come of age with testing simply being a part of life. ESSA and NCLB are merely the latest of these kinds of policy structures. You can go back to the middle of the 20th century and find policymakers describing how dependent they are on test scores for some kind of information about what is going on in local schools. The argument traditionally made for the past 15 years against standardized-test scores is that they are inaccurate and end up being harmful, but there’s something missing from that: We have a better way to do this.

After finding out that your child’s school district was ranked only by SAT scores and student-teacher ratios, you set out to create a new set of tools for measuring school quality. Three years into this work, what changes have you seen at the classroom and district levels?

Educators tend to trust data more when they see that those data align with all the things they are trying to do in the classroom."

We engaged stakeholders in the community in six districts [across Massachusetts]. We talked to students, families, teachers, principals, district administrators—several hundred of them, in fact, to find out what it is they value and what they think good schools do. And then we organized all this into a school quality framework with five major categories: teachers and the teaching environment; school culture; resources; academic learning; and citizenship and well-being. We’ve found that public perceptions of the public school actually improved when people had a wide array of performance data rather than just standardized-test scores, which tend to correlate strongly with family income. We’ve also found that educators tend to trust data more when they see that those data align with all the things they are trying to do in the classroom.

How can more expansive ways of measuring schools be helpful to educators specifically?

They won’t be put in this position where they are being asked to do a job that is different from the job they believe in. Even just to shift from standardized testing to performance assessment (a broader and more comprehensive way of measuring student learning), what we see is teachers feel more empowered. They feel like they have a bit more autonomy to act as professionals.

Some of the data we’re producing are enabling teachers to do some experiments in their classroom, where they want to figure out how to engage students better, how to build stronger relationships with students and [greater] trust in the classroom. These are ultimately the kinds of data points that improve instruction and give educators information they didn’t already have, as opposed to standardized-test reports that will tell educators students are performing below a level of proficiency. An educator who spent 180 days in the classroom with that child probably knew that already.

Test Scores Don’t Tell the Whole Story School success is often measured by test scores, but they don’t paint an accurate picture, says Jack Schneider.

It’s both about the spirit of a school and helping to restore the full purpose of education, and it’s about actionable information. For students and parents and citizens who are interested in public education, it’s about getting better at all of the things that we want to get better at, not just in this one narrow area—the acquisition of the academic content in math and English—that often produces unintended consequences like undermining the rest of the curriculum.

You argue that testing results are often more indicative of students’ socioeconomic status than a school’s quality, and that a testing culture creates stigmas harmful to student success—particularly for low-income students. How so?

Students who start with all kinds of early advantages—not just material advantages, but the academic advantage of having two parents who are college-educated and are emphasizing education in the household and setting a tone and example—those carry into school. And students who have those parents will do better on standardized tests than their peers who don’t grow up in such households. When we look at proficiency scores, we are often seeing the effect not of the school, but of the household on that individual child. Quality-conscious parents with the resources to move, to choose a “good” school or school district, begin to move to particular districts and choose particular schools and bring with them their high-scoring students. And the opposite is true when the school is branded as low-performing. One way to break this cycle is to begin to measure things that do not align so closely with demographic variables.

You say that you want to create better ways to measure schools, not just to better evaluate them individually, but to rethink as a nation how we conceptualize what a “good” school is. How has your definition of a good school been altered?

Most of us in our hearts know what a good school is and does, but we don’t have the language ready and we haven’t conceptualized it as a broader community. The way I tend to think about school quality these days is shaped by the future. Who are the kids I want to meet 10 years from now? That is not a world that is going to be made by drill-and-kill instruction in math and English. It’s a world that is going to be shaped by children discovering their interests and passions and talents in the classroom. It’s shaped by children going to school with people who are not always exactly like them and learning how to work together and who are given opportunities to play and create together. We are limited right now by our policies, tools, and language, but those things can change. We are not limited by our vision or by our shared sense of community.

The interview has been edited for length and clarity.

Related Tags:

Follow the Education Week Commentary section on Facebook and Twitter.
A version of this article appeared in the September 06, 2017 edition of Education Week as What Makes a School Good? It’s More Than Test Scores

Events

School & District Management Live Event Education Week Leadership Symposium
Education Week's Premier Leadership Event for K12 School & District Leaders.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Law & Courts Webinar
The Future of Criminal Justice Reform: A Sphere Education Initiative Conversation
America’s criminal justice system is in crisis and calls for reform are dominating the national debate. Join Cato’s Sphere Education Initiative and Education Week for a webinar on criminal justice and policing featuring the nation’s
Content provided by Cato Institute
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
Equity, Care and Connection: New SEL Tools and Practices to Support Students and Adults
As school districts plan to welcome students back into buildings for the upcoming school year, this is the perfect time to take a hard look at both our practices and our systems to build a
Content provided by Panorama Education

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment Opinion What Federally Mandated State Tests Are Good For (And What They Aren’t)
Spring 2021 testing is happening. That can be a good thing—if the goal is about more than school accountability.
Stuart Kahl
5 min read
Two people analyze test data
Visual Generation/iStock/Getty
Assessment Opinion The National Assessment Governing Board’s Troubling Gag Order
NAGB's recently released restrictions on how its board members can communicate set a troubling precedent.
3 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Assessment Whitepaper
INVALSI Addresses Italy’s COVID-19 Learning Loss
Find out how INVALSI worked with TAO to develop a plan of action that can serve as a model for other education leaders grappling with the...
Content provided by TAO by Open Assessment Technologies
Assessment Biden Administration's Level of Tolerance for Cutting Standardized Tests Comes Into Focus
A distinction has grown between states having to make tests available, and districts deciding it's not practical to make students take them.
8 min read
Image of a test sheet.
sengchoy/iStock/Getty