Graduation-Rate Plans Called All Over the Map

A new report details the patchwork of ways in which states plan to count graduates under the No Child Left Behind Act, raising concerns among some experts about a crucial part of the federal law that has received scant attention.

Set to be released this week, the paper by Christopher B. Swanson, a research associate at the Urban Institute, a Washington think tank, shows how each state intends to meet the law’s provision mandating that high schools be held accountable for the percentage of students they’re able to graduate. The result is a highly varied landscape.

“I suspected that there would be significant variation, because it appears that the states are allowed a lot of flexibility,” Mr. Swanson said. “You do see very different ways of approaching this.”

To draw his map, he surveyed the compliance plans that all 50 states—plus the District of Columbia—submitted to the U.S. Department of Education. What he found was that 45 states, along with the District of Columbia, described one of four distinct approaches. The rest he categorized as “other.”

Each technique has its own strengths and weaknesses, Mr. Swanson said.

The report, “NCLB Implementation Report: State Approaches for Calculating High School Graduation Rates” is expected to be available online October 1, from the Urban Institute.

Thirty states plan to use a formula devised by the National Center for Education Statistics that computes graduation rates by figuring in four years of dropout data.

But while that method allows for some consistency, many experts consider dropout rates unreliable, given how hard it can be to tell whether missing students have dropped out or transferred. The recent flap over dropout data in Houston underscores that point. (“Houston Case Offers Lesson on Dropouts,” Sept. 24, 2003.)

Apples and Oranges

In addition to those states using the NCES approach, Mr. Swanson noted that a few states in his “other” category described formulas that used dropout rates in some way.

Two states—Louisiana and New Jersey—plan to actually use dropout rates, instead of graduation rates, on an interim basis while phasing in other procedures, he said.

“There are a lot of states that are using dropout rates to inform their graduation rates,” Mr. Swanson said. “It is a very big issue, and a little bit under the radar screen.”

Getting around some of the problems posed by dropout rates, 10 states have chosen to use a “longitudinal” rate, in which they follow cohorts of students as they go through high school to see who among them graduates in four years.

But doing so requires a system that can track individual students as they move in and out of different schools, which is something many states don’t have.

A handful of states plan to use a simpler “completion ratio,” calculated by comparing the total number of students who graduate from a high school with its 9th grade enrollment three years earlier. Although eliminating the need for a large tracking system, such a ratio can be skewed by student mobility and by policies that hold some students back in certain grades.

In many places, the report points out, the methods are in flux. Many states, for example, plan to put into place a system for tracking students longitudinally in the next few years.

The new study builds on earlier work by Mr. Swanson. Last winter, he co-wrote a paper that showed how different formulas can yield different graduation rates for the same school district. (“Study: Formulas Yield Widely Varied Graduation Rates,” May 21, 2003.)

Using a type of completion ratio, for instance, he calculated a graduation rate of 71.7 percent for Georgia’s Cobb County schools, while using the NCES method produced a rate of 82.8 percent.

Cause for Concern?

Some analysts see reason for concern in the results of Mr. Swanson’s latest study.

Daniel J. Losen, a policy expert with the Civil Rights Project at Harvard University, said federal education officials were violating the intent of the No Child Left Behind law by giving states too much wiggle room in deciding how to count their graduates.

Mr. Losen pointed out that Congress warned of the problems posed by using dropout rates in calculating graduation rates. In drafting explanatory text that accompanied the 2001 law, federal legislators said that whatever method is used must “avoid counting dropouts as transfers.”

The law defines a graduation rate as “the percentage of students who graduate from secondary school with a regular diploma in the standard number of years.” Those with General Educational Development certificates are not to count as graduates.

Based on such language, Mr. Losen argues that to pass muster, an approach toward calculating graduation rates must track students over time as they go through school, as does the longitudinal rate described in the Urban Institute paper.

Formulas that rely on dropout rates to indirectly compute graduation rates—like the NCES method— do not, he added.

In guidelines sent to the states, the Education Department suggested using the NCES method, although the agency never required it.

Having an accurate and consistent method is critical, Mr. Losen contends, to tell if some schools are raising their achievement levels merely by pushing out their lowest-performing students.

“If states look the other way, or if this is not calculated correctly, then you will see this problem of ‘push outs’ get worse over time,” he said.

Officials with the Education Department argue that it wouldn’t have worked to demand that all states follow the same formula in computing graduation rates. Not only do states vary widely in their ability to track students, but many experts agree that the field of education lacks consensus on what is the best approach for calculating graduation rates.

“I think all the states are eager to find the most accurate way to do this,” said Celia Sims, a special assistant in the department’s office of elementary and secondary education. “There’s just no silver bullet right now.”

In the meantime, Mr. Swanson believes it’s important scrutinize the numbers once states begin spitting out rates based on the formulas they have chosen.

So much of the discussion of the No Child Left Behind Act has focused on test scores, he said, that it would be easy to miss problems with graduation figures.

“This is not an issue that’s gotten much attention,” the researcher said. “And it turns out to be a lot more complicated than people thought.”