The decision to revamp an old testing program or to start a new one is the easy part for states looking to make educators more accountable for student achievement.
The hard part is getting parents, politicians, and educators to agree on the best way to measure what students know and are able to do.
Many states that are putting in place new testing programs or overhauling old ones are caught up such conflicts: What types of questions best measure student achievement? Is the test going to positively influence classroom practice and learning? How do the decisions play politically across the state?
Answering these questions often boils down to whether to give a student a choice between a fill-in-the-bubble multiple-choice sheet or the open spaces of a blank page. Advocates of less expensive multiple-choice tests say children need to master basic skills. Backers of open-response test items say such a format better tests children’s critical-thinking and problem-solving ability.
States Debate Mix of Questions
Such debates are raging in states across the nation. Last week in Indiana, legislators and state officials apparently struck a deal on legislation that would set up a panel to oversee the kinds of questions asked on the controversial statewide test there.
Next week, Kentucky’s state school board will hear a presentation about the contractor’s new bid to run the state’s revamped testing program, which includes multiple-choice questions for the first time. The board could make a decision on the bid next week.
State officials who are considering redesigning their testing programs are closely watching the actions of some of their predecessors. Caution is increasingly the watchword, testing experts say, given the experiences of places like California and Arizona. Both states recently dumped troubled statewide testing programs that they are now laboring to replace.
And states grappling with what is the best mix of multiple-choice and so-called performance-based questions for large-scale student assessments are getting mixed signals.
While many experts tout the value of performance items, the National Assessment Governing Board in Washington, which sets policy for the National Assessment of Educational Progress, is considering including a greater proportion of multiple-choice items in the test known as the nation’s report card. Performance items--essays or hands-on experiments--are more expensive than multiple-choice tests to develop, administer, and score. (See Education Week, Jan. 24, 1996.)
Intrusive in Indiana?
In Indiana, the types of questions that should appear on a statewide assessment have been the focus of contention.
The state in March is to administer its first tests with an applied-skills part containing short-answer and essay questions. To get to that point, however, the state has had to surmount legislative and legal hurdles.
An attempt last year to replace the state’s standardized testing system was met in the legislature with strong--and successful--opposition. (See Education Week, April 12, 1995.)
Critics said the new essay and short-answer questions made the scoring of the proposed test too subjective. Cost was a factor, too. Even when the program’s $100 million price tag was slashed in half, it could not garner enough support.
Later, the legislature passed a revised version of the current Indiana Statewide Testing for Educational Progress, called ISTEP-Plus. It is largely a multiple-choice exam.
ISTEP-Plus carries a price tag of $24.8 million a year, which includes money to re-educate students who fail to meet the standards for what essential skills they should have, said Mary Tiede, a spokeswoman for the state education department.
But that test, too, came under fire.
Four state legislators and two parent groups filed a lawsuit in September against the education department seeking to block the revised assessment. A judge refused to stop the test from being administered, saying the legislature should settle the dispute.
The suit argued that some open-ended questions requiring written answers and student opinions were too intrusive.
“Open-ended is not a problem. Essay questions are not a problem,” John R. Price, a lawyer for the plaintiffs, said last week. But he said his clients did not want “essay questions that are intrusive and psychologically probing in nature.”
However, Ms. Tiede said, the questions are not designed to elicit a specific kind of response from students.
“The questions are designed to really engage a student, so that they’re able to fully display their talent in reading comprehension and writing,” she said.
“Personal opinions were never part of the scoring process,” Ms. Tiede said. “What they write, how they are able to express themselves ... that’s how those answers are graded.”
‘Judgment Call’
In Kentucky, the mix of questions was just one issue state officials have grappled with in recent months as part of a series of changes for their groundbreaking statewide student assessments--the Kentucky Instructional Results Information System.
KIRIS is central to the state’s landmark 1990 education-reform act, which essentially rebuilt the state’s entire education system.
The battery of tests covers several academic subjects and asks students to demonstrate their knowledge not with multiple-choice questions but with essays, physical tasks, and portfolios of accumulated classwork.
The state bases conclusions about the effectiveness of teachers on the assessment and passed out $26 million in cash rewards to high-scoring schools last year.
Officials have been making changes to the test battery since it was first administered in 1992. But last year two independent evaluations criticized some features of the testing program, prompting thoughts of more changes. (See Education Week, Feb. 22 and July 12, 1995.)
Last fall, Kentucky needed to decide what changes to make on KIRIS because it was time to seek bids for the testing contract, which will expire this year.
Officials found that teachers, parents, and others had concerns about the testing system as a whole and, in particular, about the lack of multiple-choice questions, which could show where students stand when compared with a national norm.
“Certainly we want to be responsive to the concerns of people we’re trying to serve,” said Ed Reidy, a deputy commissioner in the Kentucky education department who oversees the testing program.
But the state refused to take up a recommendation from testing experts to abandon student portfolios--whose scores were considered too unreliable--for accountability purposes.
“You have a number of teachers and educators who will argue the portfolios are the most important thing to improve their teaching,” said Robert F. Sexton, the executive director of the Prichard Committee for Academic Excellence, a Kentucky citizens’ group. “And it’s a judgment call of weighing the difficulties of using them in evaluation against the benefit for improved instruction.
Daniel M. Koretz, a senior social scientist with the RAND Corp. in Washington, praised Kentucky for taking bold steps in revamping the test and its format.
“They did acknowledge the need for an audit mechanism to look for score inflation ... [and] to look basically for any kind of bias,” Mr. Koretz said. “It puts them ahead of most states.”
Back to Basics
In contrast to states that are expanding their use of performance-based tests, the Idaho board of education last November approved a second year of the state’s expanded standardized testing program.
This past fall, the number of grades taking the norm-referenced, standardized Iowa Test of Basic Skills, or ITBS, tripled from three to nine, encompassing grades 3-11.
The nod from the state board was a win for the conservative Republican state schools chief, Anne C. Fox, who was elected in 1994 on a back-to-basics platform.
In an interview last week, Ms. Fox said the standardized tests and the feedback they provide to teachers will improve instruction.
She called the standardized tests an excellent measure of student mastery of basic skills and, by extension, a useful accountability tool so that state residents know how their money is being spent.
The fall 1995 test expansion cost $500,000, and Ms. Fox expects the second year to cost the same, primarily for the individual test scores for each student.
But the Idaho Education Association intends to fight legislative funding of that second year of testing, said the teachers’ union president, Monica Beaudoin. She said standardized tests tell educators little about students’ mastery of academic skills or how teachers can improve instruction.
Ms. Beaudoin said she has heard reports of teachers “teaching to the test” in the case of the ITBS.
But, Ms. Fox, said, “Teaching to the test is fine with me as long as they’re teaching the basic skills that children ought to master.”
“You do need drill and practice,” Ms. Fox said.
