After years of experimentation and funding problems, California is poised to put in place an ambitious redesign of its student-assessment program.
Beginning this spring, nearly every student in grades 4, 8, and 10--a total of 1.1 million students statewide--will be tested in reading, writing, and mathematics on new examinations that will be primarily performance-based.
Next year, the assessment will be expanded to include history and science for grades 5, 8, and 10; in the future, it will also include student portfolios and “curriculum embedded’’ elements, such as projects.
Other states, notably Kentucky and Vermont, have already implemented alternatives to multiple-choice tests as part of a national move to assessments that more closely reflect the type of instruction educators want to foster. But California’s program has attracted national attention for several reasons.
For one, it is part of a larger effort at what now is called “systemic reform’': an attempt to tie together reforms in curriculum, teacher training, and assessment so that all pieces are aimed in the same direction. In the past, state officials note, California has attempted to revamp instruction by releasing curriculum “frameworks’’ and creating professional-development programs tied to them, but these efforts were thwarted because the assessment program did not match the frameworks’ goals.
The new program is also being watched closely by educators nationwide because of its sheer size. As the largest state, California is testing whether such innovations can be implemented on a large scale, observed Edward D. Roeber, the director of the student-assessment compact at the Council of Chief State School Officers.
“Things that may be feasible in a small New England state can get costly in a large state,’' he said. “Just because you have 20 times the number of students, the public will not give you 20 times the amount of time to get it done.’'
Teachers and administrators who gathered here last month for an “awareness conference’’ called to inform them about the new program warned that state officials have their work cut out for them. The state, they pointed out, will have a massive job retraining teachers to teach the way the new assessment demands, and a huge task of creating a system that relies heavily on teacher judgments and that is also technically sound.
“To get from where we are to fully implementing [the new system] is going to be a challenge,’' Tony Anderson, the director of student information for the Fullerton school district, said.
But state officials say they are confident they can pull off the new program. They note that they have several advantages in their favor, including an existing teacher-training network, and that, in an unprecedented move, they have enlisted the four major commercial test publishers as contractors to develop the assessments.
They also point out that, at least so far, Gov. Pete Wilson and the legislature appear willing to provide the needed funds for the program, despite severe fiscal constraints. State funding for the assessment program had been cut off under Mr. Wilson’s predecessor.
“We are very optimistic the problems can be solved, but not easily,’' said Dale C. Carlson, the director of the student-assessment system for the state department of public instruction. “But our commitment is resolute. We don’t see any alternatives on the horizon but this.’'
Pioneering Work Cut Short
Although it is not the first state to implement alternative assessments, California was one of the pioneers in experimenting with the new methods.
In 1987, the California Assessment Program became one of the first statewide testing programs to include a direct-writing assessment, and the following year it began to add open-ended questions to the math assessment.
The state under a 1990 law also provided nearly $1 million to local consortia of districts to pilot new types of assessments.
But California’s efforts to expand the use of alternative measures came to an abrupt halt in 1990, when then-Gov. George Deukmejian, amid a feud with Superintendent of Public Instruction Bill Honig, cut off funding for the CAP program, one of Mr. Honig’s priorities.
Sally Mentor, a deputy superintendent of public instruction, said Mr. Deukmejian’s action actually proved to be a blessing in disguise. Because state officials did not have to administer the testing program for a year, she said, they were able to devote their energies to planning for its redesign.
“We were fortunate the money came in slowly,’' she said. “Development work takes a long time.’'
Legislative Overhaul
As part of the development work, Mr. Honig appointed a panel, led by Thomas W. Payzant, the superintendent of schools in San Diego, to design a new assessment program. Based largely on that panel’s recommendations, along with proposals from Governor Wilson, who was elected in 1990, the state legislature in 1991 approved a measure creating the new system, which as yet does not have a name.
The legislation, known as SB 662, calls for:
- Reducing the amount of testing. Under the law, students in grade 4 will be tested in reading, writing, and math; in grade 5, in history and science; and in grades 8 and 10, in all five subjects.
- Reporting results according to the proportion of students meeting proficiency standards, rather than by comparison with other students.
- Providing scores for every student, a top priority of Governor Wilson. As part of this provision, the state will, when the system is fully in place, no longer exclude those with limited-English proficiency or disabilities from the testing program.
- Developing end-of-course tests in all major subject areas in middle and high school.
- Setting criteria for other tests, such as those used by districts.
Susan K. Burr, an aide to Sen. Gary K. Hart, the chairman of the Senate Education Committee and the sponsor of SB 662, said the measure was aimed at holding schools accountable for results at a time when the state is considering relaxing regulations on them.
“The price for additional flexibility,’' she said, “is a stronger accountability system, one that is more meaningful to teachers and school administrators.’'
Systemic Reform
Ms. Burr also noted that the new system will help the state achieve its goal of systemic reform by linking the assessment more closely to the state’s curriculum frameworks.
The frameworks, which have won praise both inside and outside the state, outline visions for reshaping instruction in major subject areas. But they have not been universally adopted in classrooms.
By matching the frameworks, Ms. Burr and others said, the assessment program will help insure that teachers employ the type of instruction the frameworks encourage--enabling students to use their knowledge and solve problems, rather than simply memorize facts.
“Those waiting to do the frameworks until the CAP matches the frameworks, your wait is over,’' said Fred Tempes, an associate state superintendent of public instruction.
Mr. Tempes and others also noted that other aspects of the state education system, which also point in the same direction, will support teachers in their shift toward instruction in the way the frameworks envision.
The state’s subject-area projects--a network run by the University of California system that trains teachers according to the frameworks in 77 sites throughout the state--can provide professional development geared to the assessment program, according to Robert Polkinghorn Jr., the director of the California Subject Matter Projects.
At the same time, said Susan Bennett, the acting administrator for planning and coordination in the student-assessment office, program-quality reviews--a new state initiative to evaluate individual school programs--can also help teachers revise their instruction along the same lines. Such reviews, Ms. Bennett noted, will be based on student performance on the assessments.
“It’s beginning to come together,’' she said.
Mixed Signals?
But teachers and administrators at the Los Angeles conference cautioned that not all programs are pointed in the same direction, and that the conflicting programs might send schools mixed signals.
The state, for example, continues to require students to pass a traditional multiple-choice high school proficiency test in order to graduate.
And federal Chapter 1 rules require data from nationally normed tests, Mr. Anderson, the Fullerton school district administrator, pointed out.
“We continue to administer the Comprehensive Test of Basic Skills year after year,’' he said. “That’s the prototypical instrument they are talking about getting rid of.’'
Moreover, the new state assessment system thus far does not completely match the frameworks. Despite pleas from educators, the assessment this year includes some multiple-choice questions, Mr. Carlson of the state education department said.
“We hear, ‘Some things are efficiently measured by multiple-choice,’ or ‘That’s bologna; throw them out,’ '' the assessment director said. “We listened to both.’'
But he said the other elements of the assessment system--the portfolios and the curriculum-embedded component--will reduce the mismatch between the curriculum and assessment, since they will measure students’ performance based on their classroom work.
And Mr. Carlson predicted that other tests, including commercial tests, would also move toward the goals of the California assessment system.
“Most everybody, by the time we get to this, will be doing it right,’' he said.
Obstacles to Goals
But while teachers and administrators here said they share the goals of the new system, they pointed out that the state has a long way to go to achieve them.
Perhaps the largest obstacle, they suggested, is the fact that most teachers are not prepared to teach the way the new assessment demands.
“For a lot of teachers, it’s the opposite of the way they learned in college, the way they have been doing things,’' said Madeline Spencer, a 4th-grade teacher at the Erwin School in La Puenta.
Educating teachers in the new methods will take a substantial amount of time and money for staff development, she said.
She also warned that the state may face resistance from teachers reluctant to change their ways.
“I bet a lot of teachers take early retirement,’' Ms. Spencer said.
Other educators here noted that the size of the program represents a significant hurdle. Because SB 662 mandates scores for every student, the cost of the program will be a great deal higher than for the old CAP, which used a “matrix sampling’’ method and reported only schoolwide scores.
Moreover, performance-based assessments cost considerably more than multiple-choice tests, which can be scored quickly by a computer.
Ms. Burr of Senator Hart’s staff pointed out, though, that state officials appear willing to provide adequate funding for the program; in fact, Governor Wilson this year proposed an additional $12.5 million for it, which would bring total funding to $28.5 million.
“Any time new money is proposed, the issue is discussed a great deal’’ in the legislature, Ms. Burr said. “But there is very strong support for the program.’'
Publishers Enlisted
But even if the funds are forthcoming, state officials acknowledge that they also face technical hurdles in building the system.
Although several states have implemented assessments using performance tasks, none has yet done it on the scale of the new California program, particularly with the requirement for individual-student scores, Mr. Roeber of the state chiefs’ council said.
“It’s very easy to develop a group profile, and talk about a school, a district, a state,’' he said. “To reduce it to individual student scores, with enough technical quality, is a challenge they’ve got for themselves.’'
Moreover, Mr. Roeber noted, only one state--Vermont--has implemented portfolios, and none has yet put in place an assessment system based on curriculum-embedded work. Both of these methods raise substantial technical hurdles; a recent report by the RAND Corporation found that, in Vermont, the “rater reliability’'--the extent to which raters agreed on the quality of a student’s work--was too low to report any results below the state level. (See Education Week, Dec. 16, 1992.)
But Mr. Carlson, the California assessment-program director, said that some of the leading experts in the country are at work to try to perfect the innovative features of the project.
In an unprecedented move--unplanned, Mr. Carlson said--the state has hired the four major commercial test publishers to develop the new instruments. The companies are CTB Macmillan/McGraw-Hill, the Educational Testing Service, the Psychological Corporation, and the Riverside Publishing Company.
Mr. Carlson also expressed optimism that the technical hurdles can be surmounted. The issue of using classroom work as part of an external assessment system, he said, remains “an open question.’'
“There’s no evidence that you can’t,’' he said. “By and large, it hasn’t been tried.’'
“But you can’t study it unless you have it there,’' Mr. Carlson said.
