“Growth models” are being held out as a better way to hold schools accountable than the method the No Child Left Behind Act uses now. But in an experiment with such models, they don’t appear to be making a big difference in the proportion of schools meeting annual goals under the federal law, an analysis by the Editorial Projects in Education Research Center concludes.
Return to the main story,
Six of the nine states participating in the U.S. Department of Education’s growth-model pilot project provided information to the EPE Research Center detailing the number of schools making adequate yearly progress, or AYP, under the growth model. Under a growth model, states measure individual students’ progress instead of comparing test scores of a cohort of students against those of students in the same grade during the previous year.
In most cases in the pilot states, only a small percentage of schools made AYP using the growth model alone. Most schools that made AYP under the growth model also hit the law’s achievement targets under the standard accountability model used in most of those states.
Accountability experts and state education officials attribute the relatively small impact of growth models in part to the stringent structure that the Education Department has established for states participating in the pilot project.
“The department has pretty tough rules on what qualifies for growth models … to make sure that this wasn’t just a free ride for schools,” said Eric A. Hanushek, a senior fellow at the Hoover Institution, based at Stanford University, who has advised the department on the issue.
The growth models in the pilot program must follow a set of “bright line” principles under the pilot project. For instance, the accountability plans must adhere to the NCLB law’s target that all students be achieving at the proficient level on state tests by the end of the 2013-14 school year. Like standard state accountability plans approved under the law, also referred to as “status models,” acceptable growth models must hold schools accountable for the performance of subgroups of students, such as racial and ethnic minorities or students with disabilities.
State officials and accountability experts point out that many states use statistical techniques in their status models to ensure that schools aren’t improperly labeled as not making AYP. Such techniques are generally barred under growth models. That distinction may help explain why some schools made AYP under the status model, but not under a growth model, and why the use of growth models benefited so few schools in some of the pilot states.
Alaska, Arizona, Delaware, Florida, Iowa, and North Carolina all submitted information for the EPE Research Center’s analysis. Arkansas and Tennessee are also participating in the pilot project, but did not provide data to the research center in time for the analysis. Ohio has also been selected for the pilot project, but it must tweak its accountability plan before it can employ the growth model.
NOTES:
* Three states participating in the growth-model pilot are not represented in this analysis. Arkansas and Tennessee did not provide data to the EPE Research Center as of publication. Ohio may not employ its growth model until it adopts a uniform minimum group size for AYP calculations.
** Arizona and Delaware exclude certain schools from growth-model calculations based on size and/or grades served. For these states, this analysis includes only schools for which both the traditional and growth-model calculations were performed.
SOURCE: EPE Research Center, 2007
Statistical techniques used in status models include what are called confidence intervals, which permit a school, or a subgroup, to make AYP even if it misses its target, as long as its performance falls within a band set around that target, similar to the margin of error in polling data. States also employ “safe harbor” provisions, which were included in the law; those provisions allow schools credit for making adequate progress as long as more students maintained or moved up to proficiency in the current school year than in the previous one.
The Education Department does not permit states to use those techniques with their growth models, so many schools are still labeled as not making adequate progress.
“The rules are limiting the potential benefits of growth models, but it’s definitely a step in the right direction,” said Pete Goldschmidt, an assistant professor of education at California State University- Northridge.
For instance, Alaska applied its growth model both to schools that made AYP under the traditional status model and to those that did not. No school in the state made AYP under the growth model that did not qualify under the status model.
In fact, 150 schools made AYP under the original status model, but would not have under the growth model alone.
Les Morse, the director of assessments and accountability for the Alaska Department of Education and Early Development, attributed those results in part to the state’s use of confidence intervals and the safe-harbor provision. He said the state at first sought to include those techniques in its growth model, but the federal Education Department rejected the proposal.
Even though growth models haven’t changed the label of many schools, officials in participating states say they are learning from taking part in the pilot.
Growth models provide “instructionally valuable” information for schools to “really get them to think about where [an individual] kid should be next year,” Mr. Morse said.“We didn’t have that kind of information until this growth model moved us to produce it.”
Growth models also enable states to recognize teachers and schools that deal effectively with students at various levels of proficiency, said Thomas E. Deeter, a lead accountability consultant for the Iowa Department of Education.
The analysis by the EPE Research Center, which is affiliated with Education Week, shows that growth models made a more dramatic difference in some states than in others. North Carolina applied a growth model only to schools that did not make adequate yearly progress under the status model. Just 12 schools made AYP under the growth model that would not have made it under the status model, or about 1 percent of all 1,052 schools in the state that made AYP.
By contrast, about 14 percent of the schools that made AYP in Florida made it under the growth model but not the status model. Like North Carolina, Florida applied the growth model only to schools that did not initially make AYP under the status model.
Mr. Hanushek of the Hoover Institution suggested that the use of a growth model might have made more of a difference in Florida than in North Carolina because more students are already at the proficient level under North Carolina’s standards. Florida’s standards, he said, are generally more closely aligned with the National Assessment of Educational Progress, the federal program known as “the nation’s report card,” and so are more rigorous.
“That has an influence,” Mr. Hanushek said.
Florida officials said they hadn’t studied North Carolina’s accountability plan or standards closely enough to say whether they agree with that conclusion. They pointed out that growth in achievement was already a factor in Florida’s accountability system, so schools were paying close attention to individual students’ progress even before the growthmodel pilot.
“We were actually quite pleased with the impact we saw with [the growth model],”said Juan Copa, the bureau chief for evaluation and reporting at the Florida Department of Education.
