State Consortia To Join Forces To Develop Assessments
Consortia of states are preparing for the first time to jointly develop new student assessments based on commonly developed frameworks.
In requests for proposals sent out last month, the Council of Chief State School Officers has asked test publishers to bid to work with participating states to develop assessment measures in science and U.S. history.
The council is expected in the next few weeks to issue a similar request for a primary-level literacy assessment.
Although the project could result in a multi-state examination, that is not its aim, according to Edward Reidy, the director of assessment and accountability for the Kentucky Department of Education.
Rather, he said, it is intended to enable states to pool their resources to share the expensive development costs associated with creating new forms of assessment.
"Since states are concerned about developing similar forms of assessment,'' Mr. Reidy said, "it makes sense doing it in the most cost-effective way possible.''
Over the long run, said Edward D. Roeber, the director of the student-assessment consortium for the C.C.S.S.O., the project will lead to the "vertical integration'' of testing programs, in which classroom tests measure emerging national standards.
"Too often, teachers measure students on one set of outcomes, states on another, and then the National Assessment of Educational Progress comes along and measures a third,'' Mr. Roeber said. "Our goal is to develop a set of [assessment] tasks that at least measures national standards.''
'Jump-Start the Process'
Created in 1991, the chiefs' collaborative, financed by contributions from participating states, brought together curriculum and assessment specialists from various states who agreed to draft guidelines for new assessments in several subject areas.
In addition to history, science, and literacy, the collaborative is developing frameworks in reading, writing, mathematics, the arts, and workplace readiness.
Fifteen states have planned for the development of the science assessment, and 10 have planned the assessment in U.S. history. Fewer states are expected to take part in the actual development phase of the project.
Participating states were required to pay $50,000 as full partners, which includes participating in the development phase, or $5,000 as observers.
Thomas Fisher, the administrator for assessment, testing, and evaluation for the Florida Department of Education, said officials in that state agreed to participate as full partners in the science project after the legislature mandated a test in the subject.
"We don't have anything in science, and we'd like to get started,'' Mr. Fisher said. "This is a way to jump-start the process.''
He added, though, that the collaborative's product may not end up as Florida's science assessment.
"A state can participate in a project like that, and accept the conclusions of the group, and still retain its option of making modifications some time in the future,'' Mr. Fisher said.
States that have already moved ahead in developing new assessments are sharing the fruits of their work with other states. For example, states involved in the primary-literacy project are drawing heavily on California's primary-grades "learning record.''
"We're putting in more than we're getting out,'' said Susan Bennett, the acting administrator for planning and coordination for the California Department of Education's student-assessment office. "But we assume some things will come out that will be useful in future years.''
Not Duplicating Efforts
Other states have agreed to limit their participation in some projects to observer status only. Mr. Reidy said that Kentucky, for example, which is participating as a full partner in history, science, and literacy, agreed to take part only as an observer in the arts-assessment project.
"We'd like to learn as much as we can about what is going on around the country, so we don't duplicate efforts,'' Mr. Reidy said.
But despite the potential for cost savings, at least one state has been forced to pull out of the project because of financial constraints.
New Jersey had planned to take part in the science and U.S. history projects, but pulled out when the state budget was trimmed, according to Edward J. Masonis, the statewide testing coordinator for the New Jersey Department of Education.
"Our original intent was to take advantage of [other states'] experience in performance assessment,'' Mr. Masonis said, "and hope to spread the cost of developing assessments over other entities besides our own, and save in the long run.''
"It didn't work out,'' he said.
U.S. History Framework
The request for proposals issued last month states that the collaborative will develop a variety of alternative forms of assessment that measure student performance against emerging national standards in U.S. history and science.
Working with the participating states, it says, the contractor will develop multiple-choice and extended-response questions, performancetasks, projects, and portfolio assessments. The contractor will pilot-test the tasks in classrooms, score the pilot-test results, and develop scaled scores for reporting results.
For the U.S. history tasks, which are expected to be used for 10th- or 11th-grade examinations, the collaborative has drawn up a framework modeled closely on that developed for the 1994 National Assessment of Educational Progress assessment in that subject. (See Education Week, March 18, 1992.)
"In studying U.S. history,'' the C.C.S.S.O. document states, "the framework proposes that students should encounter powerful ideas and compelling stories, common and diverse traditions, economic growth and setbacks, technical innovation, philosophical debate, religious conviction, and how all these forces have been connected.''
Like the NAEP framework, the collaborative project divides U.S. history into four themes and nine historical periods, and presents questions in each. The themes are: change and continuity in U.S. democracy; the gatherings and interactions of peoples, cultures, and ideas; economic and technological change; and the changing role of the United States in the world.
The framework also states that students should be involved with a variety of primary-source materials, including written narratives, speeches, photographs, caricatures, songs, and artifacts.
"To reflect the richness of history and historical sources,'' the R.F.P. states, "the assessment should use a variety of stimulus materials appropriate to U.S. history.''
In science, meanwhile, the 10 states involved in the collaborative developed their own framework that reflects "emerging ideas of what and how students should be learning in science education.''
The result is "by no means a comprehensive framework,'' the R.F.P. states, but rather represents agreed-upon material common to most of the participating states.
It says that the assessment should incorporate scientific methods, including making observations, planning and conducting experiments, drawing conclusions, and communicating ideas.
In addition, it says, the assessment tasks should integrate everyday experiences, such as social and political events and issues.
The framework divides the material in science according to six major themes--change over time, constancy, energy, models and scale, patterns, and systems and interactions--and proposes questions and tasks that ask students to demonstrate both their acquisition and application of knowledge in each theme.
For example, the framework states, under the theme of "change over time,'' students "will apply patterns and processes of natural phenomena evident over time, to explain past events, then will relate their observations to current situations and make meaningful predictions about the future.''
Some demonstrators of such knowledge, it suggests, include giving students long-term evidence of plate-tectonic movement and asking them to predict what would be the position of the plates today if a considerable difference in the pattern of movement had occurred.