CRESST To Become More Active
In Guiding Accountability Systems
A group of national testing and measurement experts has decided to play a more proactive role in helping states shape their accountability systems for students and schools.
|
At their annual conference, held here Sept. 14-15, members of the National Center for Research on Evaluation, Standards, and Student Testing, or CRESST, described plans to create standards for education accountability systems to guide state efforts.
“Everyone agrees that accountability is not going away,” said Eva L. Baker, a co-director of CRESST, which is based at the University of California, Los Angeles. “Our particular interest is to figure out what we can do to make accountability systems as productive and useful as possible for children and for people who work in schools. The concern is what we should do upfront, rather than what we should have done.”
Toward that end, CRESST plans to create standards that could guide the development of education accountability systems, similar to the “Standards for Educational and Psychological Testing.” The latter document is considered the bible among test- makers and users, and it is frequently cited in federal statute and case law.
But while that hefty, nearly 200-page set of guidelines is geared primarily toward measurement experts, the proposed standards would be aimed at a broader audience, including state and federal policymakers, the media, and the general public.
The plan is to draft a lean set of standards that would describe the minimal characteristics of a fair accountability system in language the public could understand. While the testing standards were forged through a lengthy consensus process, CRESST plans to put the newer standards on a fast track, by seeking the endorsement of various organizations rather than unanimous agreement.
The group hopes to have an initial draft of the standards by December, followed by a vetting period in the spring, so that the guidelines are available to state, district, and federal policymakers within a year.
The proposed standards reflect a more general effort on the part of the federally financed research center to play a constructive role in the accountability debate.
“When you raise a question about accountability, it isn’t because you want to kill it,” Ms. Baker said.
“It may be because you want to save it. If you don’t raise questions, then fatal flaws may occur.”
Those flaws, which were cited repeatedly over the course of the two-day meeting, include the misclassification of students and schools, the inflation of test-score gains, tests that are not aligned with standards, and the unintended narrowing of the curriculum toward only what is tested.
“For years, the research community has been walking behind an elephant with a broom,” said Daniel M. Koretz, a senior social scientist with the RAND Corp., based in Santa Monica, Calif.
|
“Policymakers start accountability systems and, on rare occasions, we have an opportunity to go in and look at what’s going on,” he said.
Ms. Baker outlined a research agenda for CRESST that includes work on value-added assessments, which examine the progress of individual students over time; methods to validate test-score gains and identify score inflation; the creation of multiple measures of student and school performance; and ways to evaluate actual classroom practice.
Too often, “what goes on in practice seems to be attending to tests and not to standards,” said Brian M. Stecher, a senior social scientist at RAND.
He urged accountability systems to include measures of classroom practice, such as the kind of work assigned to students and the curricular and instructional emphases within classrooms.
To help make the standards more concrete, CRESST plans to work with the Denver-based Education Commission of the States and others to embed the standards into accountability-system models.
Jane Armstrong, the director of policy studies at the ECS, said her organization is forming a center on accountability and performance management to help governors, legislators, and others. The center will conduct research and help translate and disseminate existing research for policymakers. A Web site, scheduled for unveiling Nov. 1, will include information on accountability systems in the 50 states, along with the tradeoffs involved in adopting different approaches.
“As states bring on new systems, there needs to be some monitoring of the effectiveness of these models,” Ms. Armstrong said.
But others at the meeting were less sanguine about where educational accountability is headed. “It’s never too late to turn around if you’re on the wrong road,” said Edmund W. Gordon, the interim dean at Teachers College, Columbia University.
—Lynn Olson
