Find your next job fast at the Jan. 28 Virtual Career Fair. Register now.
Opinion Blog


Rick Hess Straight Up

Education policy maven Rick Hess of the American Enterprise Institute think tank offers straight talk on matters of policy, politics, research, and reform.

Education Opinion

RHSU Classic: The Trouble With Economists

By Rick Hess — February 20, 2020 1 min read

This month marks the 10th anniversary of Rick Hess Straight Up, making it a propitious time to revisit some favorites from the past decade. For each of the Top 20 which run this month, I’ve offered a quick reflection or thought as to why it remains a personal favorite.

This post was inspired by a piece written by Nobel laureate Jim Heckman insisting that some findings from a complicated, boutique preschool program demonstrate the case for a massive expansion of preschool—and that the details just aren’t that important. The disinterest in program design and how things actually work was disconcertingly reminiscent of how enthusiasts banged the drum for test-based accountability and teacher evaluation. In my view, it was yet another example of an economist insisting that a research finding from a convoluted analysis can tell us with great certainty how reforms will play out in the real world. Now, on to number 7, originally posted on July 24, 2013.

Some of my best friends are education economists. That's right. Economists have added a whole lot to the education discourse in the past decade. They've shed light on dubious assumptions and frequently brought a healthy rigor, one that was too often missing in the '80s and '90s. But economists bring pathologies of their own. These were highlighted in a letter that The Washington Post published last Sunday from Nobel Prize-winning economist James Heckman. Irate that columnist George Will had questioned the benefits of early-childhood education in a throwaway line in paragraph 15 of an op-ed, Heckman wrote, "Those benefits, quoted by President Obama this year, come from my evidence-based analysis of more than 30 years of data from the Perry Preschool program. . . . It is as good a trial for effectiveness as those we currently rely on to evaluate prescription and over-the-counter drugs." Heckman went on to specify that the return on early-childhood spending is "7 percent to 10 percent per child per year" and to dismiss Will as "indifferent to information." The thing is that unlike, say, over-the-counter drugs, it turns out that preschool programs are hard to replicate with fidelity or in such a way that each additional preschool student gets the anticipated benefit. The benefits of a therapeutic regime are fairly straightforward to replicate, when it's a matter of administering this drug, to identifiable clients, under specified conditions. It's not similarly clear just what the Perry Preschool "intervention" was. Is it just going to any preschool? (Nah. Even Heckman doesn't think so.) He thinks it's going to a "high-quality" preschool. OK. . . but what exactly does that mean, and how readily can we ensure that the preschools being funded are high quality and not low quality? That's where things get sticky. After all, as with any number of heralded pilots that have disappointed at scale, it's possible that the benefits of the boutique Perry program were due to an enthusiastic staff, high-touch attention from experts hoping to see positive results, community buy-in, the Hawthorne effect, or any number of tough-to-scale factors. Decades of evaluations on Head Start have found nothing like the benefits of the tiny Perry program. Nonetheless, seeking a universal truth, Heckman asserts that positive results from a small-scale trial, conducted half a century ago, "prove" what policymakers should do with hundreds of billions today—and never mind the political, organizational, or practical complexities of trying to replicate that success in the real world. As the number and sway of edu-economists has grown in recent years, so has my concern with the pathologies they can bring. I've always been particularly sensitive to three. First, they need outcomes they can measure clearly and precisely. This limits things pretty severely and has fueled the reification of test scores and earnings data (although, in the case of Perry, the passage of decades made it possible to track a set of more interesting indicators). In their write-ups, some economists toss in a paragraph conceding that we care about other stuff besides test scores and earnings, but (lacking data points), those disclaimers tend to fizzle away without a trace. Second, many economists have a funny habit of assuming the numbers they're playing with are unimpeachable representations of reality. Now, some terrific scholars deserve special commendation for getting their hands dirty in the process of digging down to collect data and acknowledging all the inevitable messiness. But plenty of their peers find it easier to ignore the complexities or to simply reanalyze existing data sets without ever poking into the reliability and validity of the numbers themselves. Thus, results get reported as compelling because the statistical relationships are strong, even if the underlying numbers might deserve further scrutiny. Third, economists are trying to distill universal truths, like physicists or chemists. For their data to unearth such relationships, they need the effect of A on B to be steady over time or across programs. In the real world, however, I'm not at all sure that those effects are static. So, the effects of this vocational program may not tell us very much about the effects of that program, and the benefits of completing a course labeled Algebra 1 in 1978 may not tell us much about the benefits of doing so in 2013. Yet, the entire "science" of economics is predicated on the stability of these relationships. Economists have a valuable role to play, and it's one that they're playing much more effectively today than used to be the case. But, like Heckman, it's easy for them to overestimate the finality of their findings and to extrapolate blithely without acknowledging the limits of their wisdom. It's equally tempting for policymakers or advocates who like a given finding to play the same game, treating complex econometric analyses as proof positive. We'd do better to greet these analyses with less deference, more skepticism, and more questions about the practical implications.

The opinions expressed in Rick Hess Straight Up are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
School & District Management Webinar
Branding Matters. Learn From the Pros Why and How
Learn directly from the pros why K-12 branding and marketing matters, and how to do it effectively.
Content provided by EdWeek Top School Jobs
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
School & District Management Webinar
How to Make Learning More Interactive From Anywhere
Join experts from Samsung and Boxlight to learn how to make learning more interactive from anywhere.
Content provided by Samsung
Teaching Live Online Discussion A Seat at the Table With Education Week: How Educators Can Respond to a Post-Truth Era
How do educators break through the noise of disinformation to teach lessons grounded in objective truth? Join to find out.

EdWeek Top School Jobs

Human Resources Manager
Madison, Wisconsin
One City Schools
Elementary Teacher - Scholars Academy
Madison, Wisconsin
One City Schools
Communications Officer
Chattanooga, Tennessee
Hamilton County Department of Education
Special Education Teacher
Chicago, Illinois
JCFS Chicago

Read Next

Education Briefly Stated Briefly Stated: January 13, 2021
Here's a look at some recent Education Week articles you may have missed.
8 min read
Education Obituary In Memory of Michele Molnar, EdWeek Market Brief Writer and Editor
EdWeek Market Brief Associate Editor Michele Molnar, who was instrumental in launching the publication, succumbed to cancer.
5 min read
Education Briefly Stated Briefly Stated: December 9, 2020
Here's a look at some recent Education Week articles you may have missed.
8 min read
Education Briefly Stated Briefly Stated: Stories You May Have Missed
A collection of articles from the previous week that you may have missed.
8 min read