Opinion Blog


Rick Hess Straight Up

Education policy maven Rick Hess of the American Enterprise Institute think tank offers straight talk on matters of policy, politics, research, and reform. Read more from this blog.

Education Opinion

Data’s a Tool, Not a Talisman

By Rick Hess — February 08, 2013 4 min read
  • Save to favorites
  • Print

Have been out at Stanford the past couple days. Gave a talk yesterday at the Ed School about Cage-Busting Leadership, where local heavyweights (Terry Moe, Mike Kirst, Bill Evers, Rick Hanushek, et al.) pushed me to explain how far you can really get with cage-busting. Meanwhile, my old friend and dissertation chair, Harvard’s Paul Peterson, had his own fun. Pointing out that Bill Gates has now suggested that data is the answer to all our myriad problems, he asked, why worry about this cage-busting stuff at all?

While having some impish fun, Paul was also teeing up a critical point. Cage-busting requires precision, crisp definitions of problems and solutions, and the use of appropriate data. Too often, however, data can be treated as a shortcut or a quick-fix, rather than a tool. Check out Cage-Busting Leadership for the full story. But it may also be worth checking out a piece that Harvard’s savvy Jal Mehta and I penned in the most recent Educational Leadership. In “Data: No Deus ex Machina”, we argue that, “Data expose inequities, create transparency, and help drive organizational improvement.” That said, we note that, “Data can be a powerful tool. But we must recognize that collecting data is not using data; that data are an input into judgment rather than a replacement for it; that data can inform but not resolve difficult questions of politics and values; and that we need better ways to measure what matters, rather than valuing those things we can measure.”

We note, “Too often, as we talk to policymakers, system leaders, funders, advocates, and vendors, we get a whiff of deus ex machina, the theatrical trick of having a god drop from the heavens to miraculously save the day. (The phrase’s literal meaning is ‘God in the machine.’) Like a Euripides tragedy in which an unforeseen development bails out the playwright who has written himself into a corner, would-be reformers too often suggest that this wonderful thing called ‘data’ is going to resolve stubborn, long-standing problems.”

This is not the first time we’ve been down this road. Data have long promised easy answers, sometimes with discomfiting results. Frederick Kelly created the first modern multiple-choice test in 1914. Others quickly followed suit. By 1923, more than 300 standardized scales were available.

As I’ve noted in The Same Thing Over and Over, Stanford’s iconic dean of education, Ellwood Cubberley, cheered such assessments, insisting, “We can now measure an unknown class and say, rather definitely, that, for example, the class not only spells poorly but is 12 percent below standard.” Cubberley explained, “Standardized tests have meant nothing less than the ultimate changing of school administration from guesswork to scientific accuracy. The mere personal opinions of school board members and the lay public ... have been in large part eliminated.”

In hindsight, would-be reformers have consistently overestimated the potential of data and have used new data in inappropriate and troubling ways. We’d do well to keep this in mind if we intend to do more than repeat past mistakes. In the piece we flag some key problems and suggest possible responses. Among the key points:

The Problem of Judgment: There’s a seeming confidence that data will enable schools to just do “what works,” allowing science to stand in for imperfect judgment. But this is based on a caricatured view of scientific inquiry. As any real scientist or detective will tell you, questions of which hypotheses to pursue, what data to collect, and how to interpret ambiguous information rely heavily on human skill, judgment, and expertise.

The Problem of Politics: Some reformers suggest that if we just “do what the data tell us,” we’ll be able to sidestep many messy political fights about policy and practice. This is an old hope--the spinning wheels of politically driven reform have long been an impediment to sustained school improvement, but it’s naive to think that data can or should replace political debates over the values, goals, and purposes of public schools.

The Problem of Purpose: We’re often imprecise about what kind of data to use for what purpose. Crude data designed for public accountability are now being used to manage performance in ways that were never intended. One result is “data-driven” systems in which leaders give short shrift to the operations, hiring, and financial practices that are the backbone of any well-run organization and essential to supporting educators.

We believe in the promise of data--but as a tool, not as a talisman. We offer various suggestions on this count.

Build Human Expertise: Data don’t use themselves. At one level, building human expertise entails building technical skills so educators are better able to use data. More ambitiously, it involves building a field in which analysis and inquiry are central to the work.

Create Structures to Support Data Use: Teachers need training, support, and time to use data well. Increasing sophisticated data use over time requires practice and coaching from mentors or colleagues who are experienced and skilled in using data wisely. Unbundling teacher roles to explicitly create teacher leaders with expertise is one promising path.

Collect a Wider Range of Data: Test scores are useful and important, but they’re not the only form of data. Student writing samples, videos of classroom teaching, and the number of parents who attend parent-teacher conferences are other forms. What data you should collect depends in part on what problem you’re trying to solve.

Anyway, if you’re interested, you may want to check all of this out.

The opinions expressed in Rick Hess Straight Up are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.

Events

Jobs October 2021 Virtual Career Fair for Teachers and K-12 Staff
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Data Webinar
Using Integrated Analytics To Uncover Student Needs
Overwhelmed by data? Learn how an integrated approach to data analytics can help.

Content provided by Instructure
Professional Development Online Summit What's Next for Professional Development: An Overview for Principals
Join fellow educators and administrators in this discussion on professional development for principals and administrators.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Education 'Widespread' Racial Harassment Found at Utah School District
The federal probe found hundreds of documented uses of the N-word and other racial epithets, and harsher discipline for students of color.
1 min read
A CNG, compressed natural gas, school bus is shown at the Utah State Capitol, Monday, March 4, 2013, in Salt Lake City. After a winter with back-to back episodes of severe pollution in northern Utah, lawmakers and Utah Gov. Gary Herbert will discuss clean air legislation and call for government and businesses to convert to clean fuel vehicles.
Federal civil rights investigators found widespread racial harassment of Black and Asian American students in the Davis school district north of Salt Lake City, Utah.
Rick Bowmer/AP Photo
Education Tiny Wrists in Cuffs: How Police Use Force Against Children
An investigation finds children as young as 6 and a disproportionate amount of Black children have been handled forcibly by police officers.
15 min read
Jhaimarion, 10, reacts as he listens to his mother, Krystal Archie talking with an Associated Press reporter in Chicago on Sept. 23, 2021. Archie’s three children were present when police, on two occasions, just 11 weeks apart, kicked open her front door and tore through their home searching for drug suspects. She’d never heard of the people they were hunting. Her oldest child, Savannah was 14 at the time; her youngest, Jhaimarion, was seven. They were ordered to get down on the floor.
Jhaimarion, 10, reacts as he listens to his mother, Krystal Archie talking with an Associated Press reporter in Chicago on Sept. 23, 2021. Archie’s three children were present when police, on two occasions, just 11 weeks apart, kicked open her front door and tore through their home searching for drug suspects. She’d never heard of the people they were hunting. Her oldest child, Savannah was 14 at the time; her youngest, Jhaimarion, was seven. They were ordered to get down on the floor.
Nam Y. Huh/AP
Education Gunman in 2018 Parkland School Massacre Pleads Guilty
A jury will decide whether Nikolas Cruz will be executed for one of the nation’s deadliest school shootings.
3 min read
Annika Dworet and her husband, Mitch Dworet, wipe away tears as their son's name is read aloud during Marjory Stoneman Douglas High School shooter Nikolas Cruz's guilty plea on all 17 counts of premeditated murder and 17 counts of attempted murder in the 2018 shootings, at the Broward County Courthouse in Fort Lauderdale, Fla. on Wednesday, Oct. 20, 2021. The Dworet's son, Nicholas Dworet, 17, was killed in the massacre.
Annika Dworet and her husband, Mitch Dworet, wipe away tears as their son's name is read aloud during Marjory Stoneman Douglas High School shooter Nikolas Cruz's guilty plea on all 17 counts of premeditated murder and 17 counts of attempted murder in the 2018 shootings, at the Broward County Courthouse in Fort Lauderdale, Fla. on Wednesday, Oct. 20, 2021. The Dworet's son, Nicholas Dworet, 17, was killed in the massacre.
Amy Beth Bennett/South Florida Sun Sentinel via AP
Education Briefly Stated: October 20, 2021
Here's a look at some recent Education Week articles you may have missed.
8 min read