Opinion
Accountability Opinion

How to Measure School Quality

By Jack Schneider & Anil Nathan — October 29, 2013 6 min read
  • Save to favorites
  • Print

Includes updates and/or revisions.

How do you measure school quality?

A seasoned educator can get a feel for a school in a day, though it might take a year to determine its particular strengths and weaknesses. Yet parents do not have that luxury; they cannot spend long stretches of time visiting all of the schools they might send their children to. And even if they did, many might not be sure what to look for.

Given this challenge, parents tend to rely on rougher indicators of quality—word-of-mouth and standardized-test scores. Parents weigh these two factors together and then make enrollment decisions with a relative sense of confidence. But these mechanisms are highly problematic and deserve careful scrutiny.

The most obvious issue is one of inaccuracy. A school can have a good reputation or impressive test scores because it caters to a privileged population. But, as research indicates, these high-scoring schools often add less value than some of their competitors with lesser reputations or lower scores.

We believe it is possible to give parents richer and more useful information about schools."

A second problem is the effect that these indicators, particularly raw test scores, have on the system as a whole. Because standardized-test scores offer a narrow and opaque measure of performance—a measure most useful for myopically ranking schools against each other—reliance on them has ensured a small number of winners and a large number of losers. This, in turn, has damaged the reputations of countless schools and intensified the alarmist narrative of educational decline.

Finally, these methods of gauging quality are troubling because they have promoted segregation. Schools with strong scores and reputations become objects of intense competition and ultimately propel property values upward and price out the working class. Consequently, even if schools in lower-income neighborhoods are good, working-class children remain isolated from their more privileged peers, gain less access to social capital, and enjoy fewer of the resources that privileged parents bring to bear on their children’s schools. And for their part, privileged students are denied engagement with those different from themselves.

Of course, merely demonstrating the inadequacies of these measures is not enough. Parents are desperate for information. And, insofar as that is the case, they will invariably opt for unreliable intelligence over none at all.

But what if we had better, more accurate measures?

We urge scholars and policymakers to build additional tools to help parents make more informed decisions about the schools that best suit their children."

Recently, we collaborated with The Boston Globe in designing a school-rating tool that we believe will move the conversation beyond reputation or raw test scores. Pulling from the various data available to us, we include measures that reflect something about school quality. And recognizing the fact that rating schools is an inherently subjective enterprise, our model allows parents to customize rankings based on personal values, including school culture, college readiness, and diversity.

The tool is imperfect, certainly. After all, such measures are constrained by available data, which is not necessarily what we might wish for in an ideal situation. And every state approaches data collection differently, so our model is not perfectly replicable. In Massachusetts, for instance, the state generates a student growth percentile, or SGP—a measure that gauges growth by comparing one student’s history of exam scores with those of all the other students in the state with a similar testing history. On the whole, however, our tool turns the tables on some conventional thinking about school quality in the state, and we believe it will change the way parents make decisions.

So what are our measures?

The first two are drawn directly from the state using SGP for English/language arts and mathematics. Such information, we recognize, is not currently available in all states. But many are moving toward a model that does a better job of leveling the playing field between groups with different background characteristics.

Our third and fourth categories are “school climate” and “college readiness.” For the former, we use graduation and dropout rates, proxies for student and adult commitment to the process of education, as well as high school seniors’ college plans. For the latter, we use SAT writing scores and the percentage of students scoring 3 or higher on Advanced Placement tests, two relatively strong predictors of college success. Such data are widely available and, therefore, this aspect of measurement is easy to reproduce.

Lastly, we include “school resources” and “diversity.” While research is mixed about the exact impact of money, it is clear that greater resources can afford a wider range of opportunities for students. And while not all parents value diversity in schooling, it is a factor that many consider invaluable in the process of education. Our measure for school resources (expenditures per student) is straightforward. But in order to measure diversity, we have to be more creative. We imagine a level of “perfect diversity” for schools and then calculate the distance between a school’s actual population and that ideal.

BRIC ARCHIVE

Again, our tool is a limited one. Graduation from high school and intent to pursue higher education are hardly ideal proxies for school climate. Our measure of college readiness relies, imperfectly, on standardized-test scores. We have an imperfect picture of school resources, tracking how much money is spent but not how it is spent. And our diversity calculation might be criticized as somewhat arbitrary: It is based on a scenario in which white, African-American, Latino and Hispanic students, and students of other racial or ethnic groups each make up 25 percent of the school’s population. We freely admit all of this, and we have encouraged our critics to join us in advocating for the systemic collection of richer and more varied data at the district or state levels.

But despite these limitations, we believe this is the first step toward creating a better source of information for the public. As such, we urge scholars and policymakers to build additional tools to help parents make more informed decisions about the schools that best suit their children.

Beyond providing better information to parents, however, our interest lies in restoring some breadth and sanity to the way we evaluate schools. Americans are bombarded by policy talk framing most schools, particularly those not located in leafy suburbs, as abysmal failures. This disaster narrative is largely sustained by measures of quality that align neatly with wealth and position, reducing schools to competing with each other rather than being identified by their unique strengths. The most disturbing outcome of this propensity is greater inequity.

Quality-conscious middle-income parents cluster together at highly sought-after schools, seeking to better serve their own children. In the process, they not only weaken the schools they leave behind, depleting them of resources and damaging their reputations, but also, in doing so, provide a narrower educational experience for their own children. Everyone loses.

We believe it is possible to give parents richer and more useful information about schools. And, we believe it is possible to do so in a manner that builds confidence rather than eroding it. That means promoting more winners, certainly. But it also means fostering a broader sense of what it means to win, reframing our conversations about school quality to align with our true values, and laying the groundwork for an education system that is both more inclusive and more integrated.

A version of this article appeared in the October 30, 2013 edition of Education Week as Assessing School Quality

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
School & District Management Webinar
Accelerate Learning with Project-Based Learning
Earlier this year, the George Lucas Educational Foundation released four new studies highlighting how project-based learning (PBL) helps accelerate student learning—across age groups, multiple disciplines, and different socio-economic statuses. With this year’s emphasis on unfinished
Content provided by SmartLab Learning
School & District Management Live Online Discussion Principal Overload: How to Manage Anxiety, Stress, and Tough Decisions
According to recent surveys, more than 40 percent of principals are considering leaving their jobs. With the pandemic, running a school building has become even more complicated, and principals' workloads continue to grow. If we
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
Building Teacher Capacity for Social-Emotional Learning
Set goals that support adult well-being and social-emotional learning: register today!


Content provided by Panorama

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Accountability States Make It Hard to Tell How Much Schools Are Spending, Report Says
The vast majority of states aren't publishing spending data in a visually appealing or comprehensive way, according to EdTrust.
3 min read
Group of people with large pens, coins, calculator, clip board, magnifying glass and studying numbers, charts and receipts.
iStock/Getty Images Plus
Accountability Did Washington D.C.'s Education Overhaul Help Black Children? This Study Says Yes
Researchers said the district's "market-based" reforms accelerated achievement versus other districts and states.
5 min read
Accountability Opinion What Next-Gen Accountability Can Learn From No Child Left Behind
As we ponder what's next for accountability and assessment, we’d benefit from checking the rearview mirror more attentively and more often.
4 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
Accountability Opinion Let’s Make Transparency the Pandemic’s Educational Legacy
Transparency can strengthen school communities, allow parents to see what’s happening, and provide students more of the support they need.
3 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty