“Never has assessment been in a more prominent position in this country than it is right now,” Susan Sclafani, a senior adviser to U.S. Secretary of Education Rod Paige, declared at the annual meeting of the National Center for Research on Evaluation, Standards, and Student Testing, held here this month.
View presentations from the 2002 CRESST conference.
She urged those gathered for the conference of the federally financed research center, known as CRESST and based at the University of California, Los Angeles, to put the “depth of our intellect” to work crafting assessments that will help teachers improve learning.
But while attendees at the Sept. 10-11 event expressed support for the overall goals of the “No Child Left Behind” Act of 2001, they raised a number of challenges to the testing and accountability requirements in the federal law.
During a session on measuring “adequate yearly progress,” researchers asked whether it is realistic to expect all students to perform at the “proficient” level on state tests by 2014, as the law mandates.
“What’s happening around the country is people are realizing if you have a very stringent standard, you’re in hot water with regard to this law,” said Robert L. Linn, a co-director of CRESST and a professor of education at the University of Colorado at Boulder. “There are clearly incentives to lower the standard of what we’ll call proficient.”
Based on the rate of improvement students have shown on the National Assessment of Educational Progress, which tests nationally representative samples of students in key subjects, “we’ll reach 100 percent proficiency in about 110 years,” said Edward Haertel, a professor of education at Stanford University.
Mr. Haertel examined trends in state test-score data from California since 1999 to predict how many schools could fail to meet their performance targets under the federal education law over time. Even if the state sets a very low bar, he concluded, just over half the schools could be identified as needing improvement by 2014. If the bar is set high, virtually all California’s schools could fail to meet their performance targets within 12 years.
To make adequate yearly progress, the law, a revision of the Elementary and Secondary Education Act, requires schools to meet performance targets for their total student populations and for specific subgroups, such as those from racial or ethnic minorities.
|
An analysis of data by Thomas J. Kane, an economist at UCLA, found that states and schools with diverse student populations are at a particular disadvantage because of the difficulty of hitting the target for every subgroup in a school every year. “What’s going to determine failure rates between states is the percent of schools with African-American or Latino subgroups,” he asserted.
Moreover, he found little evidence that separate performance targets have the desired effect. In California, he found, black and Latino students do no better in schools that face separate performance targets for those children than in schools that do not. For the latter, he looked at schools where the number of minority youths falls just below the threshold required for reporting performance by race or ethnicity. “I do believe that incentives matter,” Mr. Kane said, but he called the subgroup rule “silly.”
Researchers recommended a number of ways to improve measures of annual progress and to reduce the volatility of test scores. Mr. Linn suggested states set a minimum size of 25 students for reporting subgroup scores, although he said there is no magic number.
He also pointed out that a flexible interpretation of the law would permit states to create an “index score” to measure progress. Such a score would give schools at least partial credit for improving the performance of students well below the proficient level. In that way, Mr. Linn said, a school’s entire attention would not focus solely on students just below the proficiency bar.
The No Child Left Behind Act also requires that, beginning this school year, states must ensure that districts annually test all limited-English-proficient students in oral English, reading, and writing. Districts must demonstrate annual progress in the number or percentage of LEP children who learn English.
But in one session, researchers argued that existing tests of English-language development are not capturing the complexity of language needed to succeed in the classroom. While most of those tests focus on the general social uses of English, they noted, they fail to pick up the more academic uses of English required in school.
Mari Pearlman, a vice president at the Princeton, N.J.-based Educational Testing Service, said that such tests should report not only on the current status of students’ English proficiency, but also suggest next steps for teachers in classrooms.
Indeed, researchers here pleaded for less focus on large-scale testing and more on classroom assessments, which are more directly tied to gains in teaching and learning.
“We have to get a general pedagogical shift regarding assessment,” said James Pellegrino, a professor of psychology and education at the University of Illinois at Chicago. Mr. Pellegrino, who chaired a committee of the National Research Council on the cognitive-science foundations for assessment, called for conceptually rich systems that would better link curriculum, instruction, and assessments so they are “deeply informative.”
Researchers at the conference also suggested ways to avoid the worst abuses of “test-driven accountability,” such as cheating and “teaching getting pushed very close to test drill.”
Daniel Koretz, a professor at Harvard University’s graduate school of education, is formulating new statistical methods that could help separate “inflated” and “real” test-score gains under high-stakes conditions. He’s also trying to design teacher surveys that would pinpoint shortcuts taken to raise test scores.
Lauren B. Resnick and her colleagues at the Institute for Learning at the University of Pittsburgh are crafting measures of “instructional quality” in classrooms. The instruments they are devising would systematically look at whether teachers are providing students with rigorous content and high levels of cognitive engagement, not just test drill. Similar work is being undertaken by other CRESST affiliates.
—Lynn Olson
