Federal

Testing Expert Sees ‘Illusions of Progress’ Under NCLB

By Scott J. Cech — September 30, 2008 5 min read
  • Save to favorites
  • Print

Harvard University researcher Daniel M. Koretz has some good news and some bad news for policymakers looking ahead to the reauthorization of the federal No Child Left Behind Act: The nation’s K-12 students are now attending Lake Wobegon schools.

Because of rampantly inflated standardized test scores, Mr. Koretz contended at a panel discussion here last week, all children seem to be above average, as in the fictional town made famous by the radio personality Garrison Keillor—or at least better academically than they actually are.

“We know that we are creating, in many cases, with our test-based accountability system, illusions of progress,” Mr. Koretz told a roomful of policymakers and educators at a Sept. 22 forum hosted by the American Enterprise Institute, a Washington think tank.

Mr. Koretz, a professor of education at Harvard’s graduate school of education who has written a new book, Measuring Up: What Educational Testing Really Tells Us, said that under the No Child Left Behind law, widespread teaching to the test, strategic reallocation of teaching talent, and other means of gaming the high-stakes testing system have conspired to produce scores on state standardized tests that are substantially better than students’ mastery of the material.

“If you tell people that performance on that tested sample is what matters, that’s what they worry about, so you can get inappropriate responses in the classroom and inflated test scores,” he said.

Mr. Koretz pointed to research in the 1990s on the state standardized test then used in Kentucky, which was designed to measure similar aspects of proficiency as the National Assessment of Educational Progress, the federally sponsored testing program often called “the nation’s report card.”

Scores on both tests should have moved more or less in lock step, he said. But instead, 4th grade reading scores rose sharply on the Kentucky Instructional Results Information System test, which resembled an NCLB-test prototype, from 1992 to 1994, while sliding slightly on NAEP over the same period.

“If you’re a parent in Kentucky, what you care about is whether your kids can read, not how well they can do on the state test,” Mr. Koretz said. “And the national assessment told us that, in fact, the gains in the state test were bogus.”

Shortage of Studies

In his book, Mr. Koretz says a similar situation occurred during the “Texas miracle” of the 1990s: Minority students posted large gains on the high-stakes Texas Assessment of Academic Skills, but much smaller ones on NAEP.

Mr. Koretz said the relative dearth to date of comparative studies on large-scale state assessments isn’t for lack of trying. He said he and other scholars have often been rebuffed after approaching officials about the possibility of studying their assessment systems.

“There have not been a lot of studies of this,” Mr. Koretz said, “for the simple reason that it’s politically rather hard to do, to come to a state chief and say, ‘I’d like the chance to see whether your test scores are inflated.’?”

Bella Rosenberg, an education consultant and a longtime special assistant to Albert Shanker, the late president of the American Federation of Teachers, echoed many of Mr. Koretz’s critiques.

“This is officially sanctioned malpractice—if we did this in health, in medical practice, we’d all be dead by now,” she said of the testing system under the NCLB law. “The fact is, we’re doing a great job of obscuring instead of illuminating achievement gaps.”

But fellow panelist Roberto Rodriguez, a senior education adviser to U.S. Sen. Edward M. Kennedy, D-Mass., the chairman of the Senate Health, Education, Labor, and Pensions Committee who helped shepherd the NCLB law through Congress in 2001, cautioned against “throwing the baby out with the bath water.”

While a more effective accountability system “needs to be a goal, I would argue, of the upcoming reauthorization of the [Elementary and Secondary Education Act],” he said, that “summative assessment is a critically important area of accountability.” The No Child Left Behind law is the current version of the ESEA, first enacted in 1965.

On the tug of war between advocates of the current system and detractors who prefer that tests better reflect the course content that students are supposed to be mastering, Mr. Rodriguez said, “I think we can have our cake and eat it too.”

Incentives for Instruction

But Mr. Koretz said some parts of the nearly 7-year-old NCLB law need to be fundamentally reworked.

“I think we ought to mandate, as part of No Child Left Behind, independent evaluations of state accountability programs,” he said. “There is nobody involved in this system who has an incentive to look for good instruction anymore—all the incentives are lined up in one direction: Increase scores on the summative tests at any costs.

“We need to create a system in which somebody … has incentives to make sure we’re not just gaming the system.”

Williamson M. Evers, the U.S. Department of Education’s assistant secretary for planning, evaluation, and policy development, who attended the panel discussion, suggested that any changes to the ESEA law should preserve its essence.

“Some are proposing changes in accountability that remove consequences; this would make it accountability in name only,” Mr. Evers said in an e-mail. “Results should be comparable statewide, or parents, taxpayers, and teachers themselves can’t tell how students are doing.”

Bruce Fuller, a professor of education and public policy at the University of California, Berkeley, praised the points Mr. Koretz makes about testing under NCLB in his book, and suggested steps to address the law’s flaws.

“Washington should first ensure that states use a common benchmark that defines proficient student achievement in reading, writing, and math,” Mr. Fuller, who is the director of the Policy Analysis for California Education research center based at Berkeley and Stanford University, said in an e-mail. “Second, neo-No Child policies should ensure that each state sets a valid way of tracking student growth over time, and provide states the resources to build necessary data systems.”

A version of this article appeared in the October 01, 2008 edition of Education Week as Testing Expert Sees ‘Illusions of Progress’ Under NCLB

Events

Recruitment & Retention Live Online Discussion A Seat at the Table: Chronic Teacher Shortage: Where Do We Go From Here?  
Join Peter DeWitt, Michael Fullan, and guests for expert insights into finding solutions for the teacher shortage.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Science Webinar
Close the Gender Gap: Getting Girls Excited about STEM
Join female STEM leaders as they discuss the importance of early cheerleaders, real life role models, and female networks of support.
Content provided by Logitech
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Achievement Webinar
Mission Possible: Saving Time While Improving Student Outcomes
Learn how district leaders are maximizing instructional time and finding the best resources for student success through their MTSS framework.
Content provided by Panorama Education

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Federal Q&A Boosting 'Pathetically Low' Teacher Pay Is Top of Mind for Bernie Sanders
The progressive senator from Vermont spoke with Education Week as he prepares to chair the Senate's education committee.
6 min read
Sen. Bernie Sanders, I-Vt., talks with reporters outside the West Wing of the White House in Washington, Wednesday, Jan. 25, 2023.
Sen. Bernie Sanders, I-Vt., talks with reporters outside the West Wing of the White House in Washington, in late January.
Susan Walsh/AP
Federal What’s Behind the Push for a $60K Base Teacher Salary
When reintroduced in Congress, a bill to raise teacher salaries will include money to account for regional cost differences.
5 min read
Teachers from Seattle Public Schools picket outside Roosevelt High School on what was supposed to be the first day of classes, Wednesday, Sept. 7, 2022, in Seattle. The first day of classes at Seattle Public Schools was cancelled and teachers are on strike over issues that include pay, mental health support, and staffing ratios for special education and multilingual students.
Teachers from Seattle Public Schools picket outside Roosevelt High School on what was supposed to be the first day of classes, Wednesday, Sept. 7, 2022, in Seattle. The first day of classes at Seattle Public Schools was cancelled and teachers are on strike over issues that include pay, mental health support, and staffing ratios for special education and multilingual students.
Jason Redmond/AP
Federal Teachers Shouldn't Have to Drive Ubers on the Side, Education Secretary Says
In a speech on priorities for the year, U.S. Education Secretary Miguel Cardona said teachers should be paid competitive salaries.
5 min read
U.S. Secretary of Education Miguel Cardona delivers a speech during the “Raise the Bar: Lead the World” event in Washington, D.C., on Jan. 24, 2023.
U.S. Secretary of Education Miguel Cardona delivers a speech during the “Raise the Bar: Lead the World” event in Washington, D.C., on Jan. 24, 2023.
Sam Mallon/Education Week
Federal A Chaotic Start to a New Congress: What Educators Need to Know
A new slate of lawmakers will have the chance to influence federal education policy in the 118th Congress.
4 min read
Rep. Kevin McCarthy, R-Calif., talks on the House floor after the first vote for House Speaker when he did not receive enough votes to be elected during opening day of the 118th Congress at the U.S. Capitol, Tuesday, Jan 3, 2023, in Washington.
Rep. Kevin McCarthy, R-Calif., talks on the floor of the U.S. House of Representatives on Jan. 3 following the first round of voting for House Speaker. McCarthy fell short of enough votes to be elected speaker in three rounds of voting on opening day of the 118th Congress at the U.S. Capitol.
Andrew Harnik/AP