Includes updates and/or revisions.
Experts enlisted to help the U.S. Department of Education shape its $350 million Race to the Top assessment competition urged the agency last week to make sure that states seeking the money share a clear vision of the testing systems they aim to design, are fully committed to the lengthy project, and carefully delineate responsibilities for getting the job done.
But exactly how the Education Department would be able to discern all that from states’ applications for the money remained unclear. And dozens of other unanswered questions cropped up at hearings Jan. 13 and 14, convened by the department to seek expert advice on how states applying for the funds should organize themselves and how states’ procurement rules could affect the process.
Authorities who testified at hearings in November and December provided feedback on other aspects of the competition, in which groups, or consortia, of states vie for money to develop common assessments aligned to common academic standards. (“Funding for Common Assessments Poses Challenge,” Nov. 18, 2009.)
Real Commitment
In this second round of hearings, Joanne Weiss, the director of the Race to the Top Fund, teamed up with Education Department colleagues to seek guidance from experts on how to write rules for the competition, which is scheduled to begin in March when application guidelines are released. The assessment grants, like other parts of the Race to the Top initiative, are financed under the federal economic-stimulus law enacted last year.
The five experts at the Jan. 13 hearing agreed that a clear governance structure for the consortia and the support of key players such as governors, universities, state legislatures, and state school boards is crucial in devising new shared assessments. They said that tests must go hand in hand with good curriculum, instruction, and professional development if they are to improve student learning.
But the advice grew less clear and less unanimous as it grappled with how the department could detect and weed out applicants who lack the capacity or commitment to undertake the lengthy, complex process of designing and implementing new testing systems.
Ms. Weiss asked how officials could figure out which consortia of states applying for the money, are truly committed to the project.
Michael Cohen, the president of the education reform group Achieve, which leads a 5-year-old consortium that crafted shared math tests for 15 states, and is also centrally involved in an effort to design common academic standards, urged the department to ask for detailed outlines of the proposed tests and of how the consortia would be organized to produce them. Federal officials should also ask applicants to describe how other parts of their states’ education systems, such as curriculum or professional development, would have to be reshaped in concert with the tests to improve student achievement, he said.
Marc S. Tucker, the president of the National Center on Education and the Economy, which oversaw a consortium to write standards and assessments in the 1990s, told Ms. Weiss that the only way to gauge that commitment is to send teams out to talk to key officials. “Whatever papers you ask people to sign in the process are worth almost nothing,” he said.
Deputy Secretary of Education Tony Miller asked how best to assess whether states had the technical and leadership capacity to develop and sustain the new testing systems.
In response, Mr. Cohen said that states should be asked to describe their technical capacity in detail, but that the Education Department should assume they will underestimate their capacity to do the job. The challenge for the department, he said, is to press applicants to be specific about their plans and capacity without forcing them to submit “works of fiction” that promise the impossible.
No Easy Answers
A management expert offered lessons learned from studying 150 consortia in various industries. Byron Auguste, a director of the management-consulting group McKinsey & Co., which studied a wide range of consortia including the Star Alliance of airlines, said no consortium has succeeded without “focus, clarity, and specificity” about its objectives and its division of labor. He offered a sobering vision of consortium prospects, however, saying that only “a few dozen” of the 150 groups studied were successful.
A recurrent question at the hearing was how states could sustain the new testing systems over time, since the grant competition is designed to support their early development and piloting.
Tim Boals, the executive director of the World-Class Instructional Design and Assessment consortium, a Madison, Wis.-based project that crafted standards and assessments for English-language learners for a group of 22 states, urged the Education Department not to presume that states can manage the tests once they’ve been developed. Like the other experts, he suggested that federal officials ask applicants to outline how they would keep the new assessment systems going longer than a few years.
Another unresolved dilemma, captured in a question by Ms. Weiss, was how much test-design detail to demand from states in their applications. Requiring too much could constrain innovation, and asking for too little might allow something less than a well-thought-out plan.
There were so few easy or complete answers to such questions that at one point, Mr. Cohen turned to Ms. Weiss and said, smiling, “If I were in your shoes, I’d be getting a little worried.”
“That happened long ago,” she said with a laugh.
