Testing Expert Sees ‘Illusions of Progress’ Under NCLB

By Scott J. Cech — September 30, 2008 5 min read
  • Save to favorites
  • Print

Harvard University researcher Daniel M. Koretz has some good news and some bad news for policymakers looking ahead to the reauthorization of the federal No Child Left Behind Act: The nation’s K-12 students are now attending Lake Wobegon schools.

Because of rampantly inflated standardized test scores, Mr. Koretz contended at a panel discussion here last week, all children seem to be above average, as in the fictional town made famous by the radio personality Garrison Keillor—or at least better academically than they actually are.

“We know that we are creating, in many cases, with our test-based accountability system, illusions of progress,” Mr. Koretz told a roomful of policymakers and educators at a Sept. 22 forum hosted by the American Enterprise Institute, a Washington think tank.

Mr. Koretz, a professor of education at Harvard’s graduate school of education who has written a new book, Measuring Up: What Educational Testing Really Tells Us, said that under the No Child Left Behind law, widespread teaching to the test, strategic reallocation of teaching talent, and other means of gaming the high-stakes testing system have conspired to produce scores on state standardized tests that are substantially better than students’ mastery of the material.

“If you tell people that performance on that tested sample is what matters, that’s what they worry about, so you can get inappropriate responses in the classroom and inflated test scores,” he said.

Mr. Koretz pointed to research in the 1990s on the state standardized test then used in Kentucky, which was designed to measure similar aspects of proficiency as the National Assessment of Educational Progress, the federally sponsored testing program often called “the nation’s report card.”

Scores on both tests should have moved more or less in lock step, he said. But instead, 4th grade reading scores rose sharply on the Kentucky Instructional Results Information System test, which resembled an NCLB-test prototype, from 1992 to 1994, while sliding slightly on NAEP over the same period.

“If you’re a parent in Kentucky, what you care about is whether your kids can read, not how well they can do on the state test,” Mr. Koretz said. “And the national assessment told us that, in fact, the gains in the state test were bogus.”

Shortage of Studies

In his book, Mr. Koretz says a similar situation occurred during the “Texas miracle” of the 1990s: Minority students posted large gains on the high-stakes Texas Assessment of Academic Skills, but much smaller ones on NAEP.

Mr. Koretz said the relative dearth to date of comparative studies on large-scale state assessments isn’t for lack of trying. He said he and other scholars have often been rebuffed after approaching officials about the possibility of studying their assessment systems.

“There have not been a lot of studies of this,” Mr. Koretz said, “for the simple reason that it’s politically rather hard to do, to come to a state chief and say, ‘I’d like the chance to see whether your test scores are inflated.’?”

Bella Rosenberg, an education consultant and a longtime special assistant to Albert Shanker, the late president of the American Federation of Teachers, echoed many of Mr. Koretz’s critiques.

“This is officially sanctioned malpractice—if we did this in health, in medical practice, we’d all be dead by now,” she said of the testing system under the NCLB law. “The fact is, we’re doing a great job of obscuring instead of illuminating achievement gaps.”

But fellow panelist Roberto Rodriguez, a senior education adviser to U.S. Sen. Edward M. Kennedy, D-Mass., the chairman of the Senate Health, Education, Labor, and Pensions Committee who helped shepherd the NCLB law through Congress in 2001, cautioned against “throwing the baby out with the bath water.”

While a more effective accountability system “needs to be a goal, I would argue, of the upcoming reauthorization of the [Elementary and Secondary Education Act],” he said, that “summative assessment is a critically important area of accountability.” The No Child Left Behind law is the current version of the ESEA, first enacted in 1965.

On the tug of war between advocates of the current system and detractors who prefer that tests better reflect the course content that students are supposed to be mastering, Mr. Rodriguez said, “I think we can have our cake and eat it too.”

Incentives for Instruction

But Mr. Koretz said some parts of the nearly 7-year-old NCLB law need to be fundamentally reworked.

“I think we ought to mandate, as part of No Child Left Behind, independent evaluations of state accountability programs,” he said. “There is nobody involved in this system who has an incentive to look for good instruction anymore—all the incentives are lined up in one direction: Increase scores on the summative tests at any costs.

“We need to create a system in which somebody … has incentives to make sure we’re not just gaming the system.”

Williamson M. Evers, the U.S. Department of Education’s assistant secretary for planning, evaluation, and policy development, who attended the panel discussion, suggested that any changes to the ESEA law should preserve its essence.

“Some are proposing changes in accountability that remove consequences; this would make it accountability in name only,” Mr. Evers said in an e-mail. “Results should be comparable statewide, or parents, taxpayers, and teachers themselves can’t tell how students are doing.”

Bruce Fuller, a professor of education and public policy at the University of California, Berkeley, praised the points Mr. Koretz makes about testing under NCLB in his book, and suggested steps to address the law’s flaws.

“Washington should first ensure that states use a common benchmark that defines proficient student achievement in reading, writing, and math,” Mr. Fuller, who is the director of the Policy Analysis for California Education research center based at Berkeley and Stanford University, said in an e-mail. “Second, neo-No Child policies should ensure that each state sets a valid way of tracking student growth over time, and provide states the resources to build necessary data systems.”

A version of this article appeared in the October 01, 2008 edition of Education Week as Testing Expert Sees ‘Illusions of Progress’ Under NCLB


Student Well-Being K-12 Essentials Forum Boosting Student and Staff Mental Health: What Schools Can Do
Join this free virtual event based on recent reporting on student and staff mental health challenges and how schools have responded.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Curriculum Webinar
Practical Methods for Integrating Computer Science into Core Curriculum
Dive into insights on integrating computer science into core curricula with expert tips and practical strategies to empower students at every grade level.
Content provided by Learning.com

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Federal Will the Government Actually Shut Down This Time? What Educators Should Know
The federal government is once again on the verge of shutting down. Here's why educators should care, but shouldn't necessarily worry.
1 min read
Photo illustration of Capitol building and closed sign.
Federal Biden Admin. Warns Schools to Protect Students From Antisemitism, Islamophobia
The U.S. Department of Education released a "Dear Colleague" letter reminding schools of their obligation to address discrimination.
3 min read
Education Secretary Miguel Cardona speaks during an interview with The Associated Press in his office at the Department of Education on Sept. 20, 2023 in Washington.
Education Secretary Miguel Cardona speaks during an interview in his office at the U.S. Department of Education on Sept. 20, 2023 in Washington.
Mark Schiefelbein/AP
Federal What Educators Should Know About Mike Johnson, New Speaker of the House
Johnson has supported restructuring federal education funding, as well as socially conservative policies that have become GOP priorities.
4 min read
House Speaker-elect Rep. Mike Johnson, R-La., addresses members of Congress at the Capitol in Washington on Oct. 25, 2023. Republicans eagerly elected Johnson as House speaker on Wednesday, elevating a deeply conservative but lesser-known leader to the seat of U.S. power and ending for now the political chaos in their majority.
House Speaker Mike Johnson, R-La., addresses members of Congress at the Capitol in Washington on Oct. 25, 2023. Johnson has a supported a number of conservative Republican education priorities in his time in Congress.
J. Scott Applewhite/AP
Federal America's Children Don't Have a Federal Right to Education. Will That Ever Change?
An education scholar is launching a new research and advocacy institute to make the case for a federal right to education.
6 min read
Kimberly Robinson speaks at the kickoff event for the new Education Rights Institute at the University of Virginia School of Law in Charlottesville, Va., on Oct. 16, 2023.
Kimberly Robinson speaks at the kickoff event for the new Education Rights Institute at the University of Virginia School of Law in Charlottesville, Va., on Oct. 16, 2023.
Julia Davis, University of Virginia School of Law