Opinion
Federal Opinion

Learning Without Loopholes

By John Merrow — December 04, 2007 6 min read
  • Save to favorites
  • Print

Congressional leaders have finally acknowledged what most observers have known for months: The No Child Left Behind Act, the signature domestic legislation of the Bush administration, will not be reauthorized in the foreseeable future. And it should not be, at least not until the law’s blatant loopholes are addressed. The truth that no one in power seems willing to admit is that the federal law encourages statistical manipulations that make reports of academic progress suspect and, in some cases, virtually meaningless.

The well-intentioned 2001 legislation declares that all identifiable “subgroups” of students (those from low-income families, members of minority groups, students with disabilities, and others) must make what it calls “adequate yearly progress.” To put pressure on educators, the law says that if just one subgroup fails to make AYP, the entire school is deemed “in need of improvement.”

Ah, but the loopholes! The law lets states decide what constitutes academic proficiency and how large a subgroup needs to be before its scores count. It also allows states to use a statistical trick that can add 50 or more points to a test score.

BRIC ARCHIVE

Because failure under No Child Left Behind has a high price tag in penalties (including loss of jobs and school closings), states would be crazy not to take advantage of whatever loopholes are available to them.

To understand the statistical trick, think of political polls. These report that Candidate X is ahead of Candidate Y, 40 percent to 33 percent, with a margin of error of 4 points. That “plus or minus” band around a score, called a “confidence interval,” makes sense, because political polls are based on a small sample of probable voters, not every registered voter.

Polls generally have a 95 percent confidence interval, meaning that 95 out of 100 times the results would not vary by more than the stated margin. To be 99 percent confident, however, requires a greater margin of error, perhaps 12 points or more, and that would mean that, in the example above, Candidates X and Y are tied.

But schools test every student, not a small sample, and so one might assume that the scores kids get are the scores they get. Not so, because schools apply a confidence interval. With chutzpah reminiscent of Milo Minderbinder in Catch-22, educators rationalize that some students might have done better under different conditions (if they’d eaten a decent breakfast, or hadn’t fought with their parents just before the test, for example). With the blessing of the U.S. Department of Education, states are allowed to calculate these hypothetical scores. And when a subgroup’s actual score falls below the target but is within the confidence interval of hypothetical scores, that subgroup is deemed to have met the NCLB requirements.

See Also

Does the NCLB law have too many loopholes? If so, can they be closed without sacrificing fairness? Join the discussion.

What a difference a confidence interval can make! Some 509 schools in Illinois—13.5 percent of those tested—failed to make AYP, but passed anyway because the state employed a 95 percent confidence interval, adding 12 “hypothetical” points.

Thanks to a confidence interval, in only 10 of 161 Kentucky elementary schools did African-American kids fail to make AYP. Without that crutch, however, 83 subgroups would have failed, which would have labeled those 83 schools failures.

Without the escape hatch provided by the confidence interval, the adults in charge would have been publicly embarrassed, but the students would have received additional resources. So, just whose interests were being served?

Parents are kept in the dark. When NCLB results are released, no asterisk tells them their children passed hypothetically but not in reality.

Thirty-six states employ confidence intervals. With the blessing of the federal Education Department, most use the 99 percent level, giving them the largest possible margin of error.

Here’s another glaring loophole: The No Child Left Behind law allows states to decide how large a subgroup will be, and the variation is staggering, ranging from a low of five in Maryland, to 50 in Texas, Virginia, and West Virginia. If a particular elementary school has 35 Hispanic students, but the state subgroup size has been set at 40, that school—as if by magic—does not have a Hispanic subgroup, and how those students perform on the state test doesn’t really matter.

High subgroup numbers let schools off the hook. By definition, it holds fewer schools accountable for educating the most vulnerable students—minority students, low-income students, students with disabilities, and students who have limited proficiency in English. Big achievement gaps are less likely to show up; schools are less likely to tell the public about them or to be pressured to do something to close them.

See Also

For regular updates on the reauthorization of the No Child Left Behind Act, read our blog NCLB: Act II.

In effect, the most vulnerable kids—the ones the law was supposed to be helping—disappear (although their scores are mixed in with the results for the entire school). The Associated Press has reported that an estimated 1.9 million minority students have “disappeared.”

Combine a large subgroup size with a confidence interval, as Wisconsin does (40 and 99 percent, respectively), and the results look impressive. In that state, only Milwaukee and Kenosha, of its 440 districts, have not made adequate yearly progress every year.

The law also lets states decide what constitutes passing. So, for example, when Illinois lowered the passing score on its 8th grade math test, the pass rate jumped from 54 percent to 78 percent.

At least seven states lowered the bar in midcourse. Others just started out with a low bar.

Policymakers had hoped that the National Assessment of Educational Progress would shame states into setting a high bar, but that hasn’t worked. On North Carolina’s own 4th grade math test for NCLB, its students had the highest passing rate in the county—92 percent. But when North Carolina students took the NAEP math test, just 40 percent passed. Similar gaps in achievement can be found in most states.

Low standards make schools look good but hurt students. To use a high-jump analogy, if the coach consistently sets the bar at 5 feet, a talented athlete will never know if he or she could have cleared 6 feet, or higher. Lowering the bar also reduces the incentives and the pressure on school districts to do a better job.

Asked if she believed states were taking advantage of the loopholes to avoid being held accountable, U.S. Secretary of Education Margaret Spellings said: “Sometimes they are, and sometimes they aren’t. It’s ‘The Naked City’; there’s lots of different answers to that question. But are some kids being left behind? You bet. Are some people gaming the system, and we ought to be watchful about that and hawkish about it? You bet. And we do our very best to do that.”

Because failure under No Child Left Behind has a high price tag in penalties, states would be crazy <i>not</i> to take advantage of whatever loopholes are available to them.

But, she added, “I choose to believe that the people in states are working hard to improve education for their kids.”

Perhaps they are, but if the No Child Left Behind law is intended to be a signaling system that sets standards, tests students, and then reveals where students are not learning what they need to know, it is sending—at best—confusing signals. At worst, the data are virtually useless.

Congress is between a rock and a hard place. If it closed the loopholes, someone in Washington would have to police all 15,000 school districts. Does anyone want the U.S. secretary of education to be the nation’s school superintendent?

On the other hand, not closing the loopholes breeds cynicism, hypocrisy, and disrespect for law.

Now a political football, NCLB is unlikely to be reauthorized before the next presidential election. The intervening months are an opportunity for states to work together to set common standards. Presidential candidates could be pressed to pledge adequate resources to allow states to develop common standards and tests. While state participation should be voluntary, once 20 or so states sign on, the rest will follow. And that would be a big step toward a healthy, internationally competitive public education system.

A version of this article appeared in the December 05, 2007 edition of Education Week as Learning Without Loopholes

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
IT Infrastructure & Management Webinar
From Chaos to Clarity: How to Master EdTech Management and Future-Proof Your Evaluation Processes
The road to a thriving educational technology environment is paved with planning, collaboration, and effective evaluation.
Content provided by Instructure
Special Education Live Online Discussion A Seat at the Table - Special Education: Proven Interventions for Academic Success
Special education should be a launchpad, not a label. Join the conversation on how schools can better support ALL students.
Special Education K-12 Essentials Forum Innovative Approaches to Special Education
Join this free virtual event to explore innovations in the evolving landscape of special education.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Federal Classroom Tech Outpaces Research. Why That's a Problem
Experts call for better alignment between research and the classroom in Capitol Hill discussions.
4 min read
People walk outside the U.S Capitol building in Washington, June 9, 2022.
People walk outside the U.S Capitol building in Washington, June 9, 2022. Experts called for investments in education research and development at a symposium at the Dirksen Senate Office Building on June 13.
Patrick Semansky/AP
Federal Opinion Federal Education Reform Has Largely Failed. Unfortunately, We Still Need It
Neither NCLB nor ESSA have lived up to their promise, but the problems calling for national action persist.
Jack Jennings
4 min read
Red, Blue, and Purple colors over a fine line etching of the Capitol building. Republicans and Democrats, Partisan Politicians.
Douglas Rissing/iStock
Federal A More Complete Picture of Immigration's Impact on U.S. Public Schools
House Republicans say a migrant influx has caused "chaos" in K-12 schools. The reality is more complicated.
10 min read
Parents and community members rally outside P.S. 189 to protest New York City Mayor Eric Adam's plan to temporarily house immigrants in the school's gymnasium, seen in the background on May 16, 2023, in New York.
Parents and community members rally outside P.S. 189 to protest New York City Mayor Eric Adam's plan to temporarily house immigrants in the school's gymnasium, seen in the background on May 16, 2023, in New York.
John Minchillo/AP
Federal Explainer What Is Title IX? Schools, Sports, and Sex Discrimination
Title IX, the law prohibiting discrimination on the basis of sex, is undergoing changes. What it is, how it works, and how it's enforced.
2 min read
In this Nov. 21, 1979 file photo, Bella Abzug, left, and Patsy Mink of Women USA sit next to Gloria Steinem as she speaks in Washington where they warned presidential candidates that promises for women's rights will not be enough to get their support in the next election.
In this Nov. 21, 1979, photo, Bella Abzug, left, and Patsy Mink of Women USA sit next to Gloria Steinem as she speaks in Washington at an event where they warned presidential candidates that promises for women's rights will not be enough to win their support in the next election.
Harvey Georges/AP