Rules Urge New Style of Testing

Competition has opened for $350 million in federal money to design new ways of assessing what students learn. Rules for the contest make clear that the government wants to leave behind multiple-choice testing more often in favor of essays, multidisciplinary projects, and other more nuanced measures of achievement.

In the final regulations for the competition, issued this month, the U.S. Department of Education says that it seeks assessments that “more validly measure” students’ knowledge and skills than those that have come to dominate state testing in recent years. It wants tests that show not only what students have learned, but also how that achievement has grown over time and whether they are on track to do well in college. And all that, the regulations said, requires assessments that elicit “complex student demonstrations or applications” of what they’ve learned.

The money for the competition is a slice of the $4.35 billion Race to the Top Fund, which is financed with money from the economic-stimulus law passed last year.

Of the $350 million set aside for new tests, the Education Department plans to award one or two grants of up to $160 million each for “comprehensive assessment systems,” and one $30 million grant that is only for development of end-of-course tests at the high school level. All grants will run for four years. Applications are due June 23, and money will be awarded in September. The department was to host a meeting in Minneapolis this week to offer help to potential applicants.

States must band together in groups, or “consortia,” of 15 or more to apply for the comprehensive-testing grant, with five states designated as “governing,” or leading, partners. Applicants for the high school testing program must show that five states are leading their group, but face no other minimum group-size requirement.

The comprehensive systems developed with the federal money must be able to yield data that can be used to gauge how well teachers and principals are doing their jobs, how instruction and school programs can be improved, and how schools can be judged for federal accountability, according to the 85-page set of regulations.

Tests must be able to measure if students are mastering a “common set of college- and career-ready” academic standards, and those standards must be adopted by the end of 2011. The National Governors Association and the Council of Chief State School Officers, with the support of 48 states, have led a move to write common standards, which are undergoing final revision. Federal officials have used states’ commitment to those standards as incentives in other programs, such as the main Race to the Top competition. (“‘Race to Top’ Standards Link Questioned,” Dec. 16, 2009.)

In the competition, states would have to put all the tests designed under that program into practice by the 2014-15 school year and figure out by then how well students would have to perform to be considered ready, at the high school level, for college or good jobs, or, in the earlier years, on track to be ready for those challenges.

Getting It Right

Jack Jennings, whose Washington-based Center on Education Policy has studied the state testing required under the No Child Left Behind Act, said it’s important to get assessment right because so much rides on it.

“We’re relying to a great degree on tests to bring about improvements in schools, but we’ve come to understand that many of these tests are not the best,” he said. “If we are going to put this much weight on assessment, we really have to get much better tests, ones that measure higher-order skills and are more useful to teachers. Unless we get better tests, we should question the basic premise of standards-based reform, which is that you can adequately measure the attainment of standards.”

Nel Noddings, a professor emeritus of education at Stanford University, said she wondered whether the effort to improve testing wouldn’t be better spent helping teachers do what tests cannot.

“You just don’t need this sophisticated assessment stuff,” said Ms. Noddings, a former math teacher. “What you need is excellent teachers who stay in contact with their kids and work with them day after day. We’ve just gone test crazy.”

Not Identical

The high school testing grant differs in some ways from the comprehensive-test program. Assessments designed under the comprehensive-test program, for instance, must yield data that can be used to judge schools’ effectiveness. They are also intended for use in federal accountability. Neither expectation applies to assessments designed as part of the high school test program.

Joanne Weiss, the director of the Race to the Top program, said the high school test competition was designed differently so it can act as a “lever for high school improvement,” encouraging states to build more rigor into their secondary school courses without the added pressure of accountability. She added, however, that high school tests designed as part of the comprehensive-test grant program would be subject to that program’s rules.

Each grant program includes incentives for states to coordinate K-12 and higher education in developing tests, something that is rarely done currently and which some advocates believe would allow schools to better line up their demands with college expectations.

Applicants for the comprehensive-testing grant program get extra points if they can show they have enlisted their four-year college or university systems to help design the tests and to agree to let entering students skip remedial work and enroll in credit-bearing courses if they score at a certain level on those tests. State consortia applying for the high school program get priority if they show their colleges or universities will help design tests that measure achievement in coursework in career and technical education, or in science, technology, engineering, and mathematics.

David T. Conley, a University of Oregon professor whose work focuses on college readiness, commended the incentives for higher education involvement.

“To use that level of common sense in designing an assessment system is unprecedented,” he said. “We tell students in high school that one of the main purposes of their education is to go on and learn beyond high school, but we never sit down together at the table, high school and college instructors, to decide what we expect and how to coordinate and align those expectations.”

But Mr. Conley warned that judging college readiness by a test- score cutoff—even on a vastly improved test—risks blowing a key opportunity to improve the way schools gauge students’ needs as they enter college. What’s necessary, he said, is a test that can show a broad range of unmet needs, from help with entry-level math to support with study skills. That information can’t come from a test with a set cutoff point, but from a variety of measurements, he said.

Sharing Content

In response to experts’ suggestions that the department use the test competition to spur and spread innovation, it is requiring states seeking the funding to commit to making test content developed with that money “freely available,” including to states not in the consortia and to commercial organizations.

Consortia’s applications will be evaluated by panels of experts assembled by the Department of Education. They will be scored on a 200-point scale, with the largest portion of points for test design and development and their plans for how research will be conducted to evaluate the tests’ validity. But they will also be judged on their plans for how their consortia will operate and how the projects themselves will be managed. Experts at public meetings held by the department to help shape the application cautioned officials that consortia—especially larger teams—can easily stumble on the massive management challenges.

Six potential assessment consortia took shape late in the winter, as states readied themselves to vie for the federal money. Those six have now merged into three, although more could emerge before the June application deadline.

Achieve, a Washington-based organization that works with states to craft standards and accountability systems, is talking with about 30 states about devising a system of summative assessments that would include performance tasks. The so-called MOSAIC consortium of states, which is working on formative assessments, is now working with what’s being called the SMARTER group, which focuses on computer-adaptive testing, and the “balanced assessment” consortium, which emphasizes a system that will include curriculum-embedded performance tasks scored by teachers.

Another consortium, headed by the National Center on Education and the Economy, aims to design high school tests modeled on the British “board exams.”