Testing Expert Sees ‘Illusions of Progress’ Under NCLB

By Scott J. Cech — September 30, 2008 5 min read
  • Save to favorites
  • Print

Harvard University researcher Daniel M. Koretz has some good news and some bad news for policymakers looking ahead to the reauthorization of the federal No Child Left Behind Act: The nation’s K-12 students are now attending Lake Wobegon schools.

Because of rampantly inflated standardized test scores, Mr. Koretz contended at a panel discussion here last week, all children seem to be above average, as in the fictional town made famous by the radio personality Garrison Keillor—or at least better academically than they actually are.

“We know that we are creating, in many cases, with our test-based accountability system, illusions of progress,” Mr. Koretz told a roomful of policymakers and educators at a Sept. 22 forum hosted by the American Enterprise Institute, a Washington think tank.

Mr. Koretz, a professor of education at Harvard’s graduate school of education who has written a new book, Measuring Up: What Educational Testing Really Tells Us, said that under the No Child Left Behind law, widespread teaching to the test, strategic reallocation of teaching talent, and other means of gaming the high-stakes testing system have conspired to produce scores on state standardized tests that are substantially better than students’ mastery of the material.

“If you tell people that performance on that tested sample is what matters, that’s what they worry about, so you can get inappropriate responses in the classroom and inflated test scores,” he said.

Mr. Koretz pointed to research in the 1990s on the state standardized test then used in Kentucky, which was designed to measure similar aspects of proficiency as the National Assessment of Educational Progress, the federally sponsored testing program often called “the nation’s report card.”

Scores on both tests should have moved more or less in lock step, he said. But instead, 4th grade reading scores rose sharply on the Kentucky Instructional Results Information System test, which resembled an NCLB-test prototype, from 1992 to 1994, while sliding slightly on NAEP over the same period.

“If you’re a parent in Kentucky, what you care about is whether your kids can read, not how well they can do on the state test,” Mr. Koretz said. “And the national assessment told us that, in fact, the gains in the state test were bogus.”

Shortage of Studies

In his book, Mr. Koretz says a similar situation occurred during the “Texas miracle” of the 1990s: Minority students posted large gains on the high-stakes Texas Assessment of Academic Skills, but much smaller ones on NAEP.

Mr. Koretz said the relative dearth to date of comparative studies on large-scale state assessments isn’t for lack of trying. He said he and other scholars have often been rebuffed after approaching officials about the possibility of studying their assessment systems.

“There have not been a lot of studies of this,” Mr. Koretz said, “for the simple reason that it’s politically rather hard to do, to come to a state chief and say, ‘I’d like the chance to see whether your test scores are inflated.’?”

Bella Rosenberg, an education consultant and a longtime special assistant to Albert Shanker, the late president of the American Federation of Teachers, echoed many of Mr. Koretz’s critiques.

“This is officially sanctioned malpractice—if we did this in health, in medical practice, we’d all be dead by now,” she said of the testing system under the NCLB law. “The fact is, we’re doing a great job of obscuring instead of illuminating achievement gaps.”

But fellow panelist Roberto Rodriguez, a senior education adviser to U.S. Sen. Edward M. Kennedy, D-Mass., the chairman of the Senate Health, Education, Labor, and Pensions Committee who helped shepherd the NCLB law through Congress in 2001, cautioned against “throwing the baby out with the bath water.”

While a more effective accountability system “needs to be a goal, I would argue, of the upcoming reauthorization of the [Elementary and Secondary Education Act],” he said, that “summative assessment is a critically important area of accountability.” The No Child Left Behind law is the current version of the ESEA, first enacted in 1965.

On the tug of war between advocates of the current system and detractors who prefer that tests better reflect the course content that students are supposed to be mastering, Mr. Rodriguez said, “I think we can have our cake and eat it too.”

Incentives for Instruction

But Mr. Koretz said some parts of the nearly 7-year-old NCLB law need to be fundamentally reworked.

“I think we ought to mandate, as part of No Child Left Behind, independent evaluations of state accountability programs,” he said. “There is nobody involved in this system who has an incentive to look for good instruction anymore—all the incentives are lined up in one direction: Increase scores on the summative tests at any costs.

“We need to create a system in which somebody … has incentives to make sure we’re not just gaming the system.”

Williamson M. Evers, the U.S. Department of Education’s assistant secretary for planning, evaluation, and policy development, who attended the panel discussion, suggested that any changes to the ESEA law should preserve its essence.

“Some are proposing changes in accountability that remove consequences; this would make it accountability in name only,” Mr. Evers said in an e-mail. “Results should be comparable statewide, or parents, taxpayers, and teachers themselves can’t tell how students are doing.”

Bruce Fuller, a professor of education and public policy at the University of California, Berkeley, praised the points Mr. Koretz makes about testing under NCLB in his book, and suggested steps to address the law’s flaws.

“Washington should first ensure that states use a common benchmark that defines proficient student achievement in reading, writing, and math,” Mr. Fuller, who is the director of the Policy Analysis for California Education research center based at Berkeley and Stanford University, said in an e-mail. “Second, neo-No Child policies should ensure that each state sets a valid way of tracking student growth over time, and provide states the resources to build necessary data systems.”

A version of this article appeared in the October 01, 2008 edition of Education Week as Testing Expert Sees ‘Illusions of Progress’ Under NCLB


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Mathematics Webinar
Addressing Unfinished Learning in Math: Providing Tutoring at Scale
Most states as well as the federal government have landed on tutoring as a key strategy to address unfinished learning from the pandemic. Take math, for example. Studies have found that students lost more ground
Content provided by Yup Math Tutoring
Classroom Technology Webinar Building Better Blended Learning in K-12 Schools
The pandemic and the increasing use of technology in K-12 education it prompted has added renewed energy to the blended learning movement as most students are now learning in school buildings (and will likely continue

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Federal Top Federal Adviser on Puerto Rico's Schools Declares: 'We Have to Build Trust'
Chris Soto heads an Education Department team providing technical assistance and support for the U.S. territory's public schools.
4 min read
Martin G. Brumbaugh School kindergarten teacher Nydsy Santiago teaches her students under a gazebo at a municipal athletic park in Santa Isabel, Puerto Rico, on Feb. 4, 2021.
Martin G. Brumbaugh School kindergarten teacher Nydsy Santiago teaches her students under a gazebo at a municipal athletic park in Santa Isabel, Puerto Rico, on Feb. 4, 2021.
Carlos Giusti/AP
Federal Schools Could Count Nonbinary Students Under Biden Proposal
The Civil Rights Data Collection for this school year could also revive questions about inexperienced teachers and preschool discipline.
6 min read
Image of a form with male and female checkboxes.
Federal 'Parents' Bill of Rights' Underscores Furor Over Curriculum and Transparency in Schools
U.S. Sen. Josh Hawley's bill highlights how education issues like critical race theory will likely stay in the national political spotlight.
7 min read
Sen. Josh Hawley, R-Mo., speaks during a Senate Armed Services Committee hearing on the conclusion of military operations in Afghanistan and plans for future counterterrorism operations, Tuesday, Sept. 28, 2021, on Capitol Hill in Washington.
U.S. Sen. Josh Hawley, R-Mo., says "it's time to give control back to parents, not woke bureaucrats."
Patrick Semansky/AP
Federal Opinion It’s Not Just the NSBA That’s Out of Touch. There’s a Bigger Problem
Those who influence educational policy or practice would do well to care about what parents and the public actually want.
4 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty