ETS Previews Revamped Examination for Teachers
Lawrenceville, N.J.--First the question pops up on the computer screen: Which members of N are multiples of 4? Below it appears the sequence (N=1,2,3,4,5,6,7,8,0,11,12,13,14,15).
On a pad at the side of the computer, the operator slides a "mouse," manipulating an on-screen arrow until it points to the answers she believes are correct. She presses a button, and a new question appears. Welcome to the new era of teacher testing, now dawning here at the 378-acre camgiant nonprofit firm is demonstrating the first phase of what will become the successor to the National Teacher Examinations, the most widely used teacher assessment in the country.
Once the multi-million-dollar overhaul of the NTE is complete, assessment officials say, pencil-and-paper testing for teacher licensure will no longer reign supreme.
Introduced in 1940, the NTE has been revised periodically, but "basically it's the test it was back then," said Gregory R. Anrig, president of the testing service. As a result of advances in technology, measurement theory, and the body of knowledge about learning, Mr. Anrig predicted, "we're going to see more changes in standardized educational testing in the "United States over the next 10 years than we have over the past 50 years."I want very much for ETS to be in the forefront," he said.
This summer, the firm put some 1,500 college students through the paces of pilot-testing the first stage of the new teacher examinations. By the fall of 1992, the ETS will begin phasing in the successor to the tests that 33 of the states use as part of their licensure requirements.
The advance look provided here was intended to keep the public and the education community apprised of progress on the overhaul project announced two years ago.(See Education Week, Nov. 2, 1988.)
Every step along the line, we're trying to involve people who will be affected," Mr. Anrig said.
The paper-and-pencil format of the NTE has drawn criticism from educators, policymakers, and members of the public, who claim it discriminates against minorities and provides insufficient or valueless data to judge an applicant's knowledge base or ability to teach.
For instance, states have had no way to gauge a fundamental skill for foreign-language teachers. "There is no way with the present NTE to test speaking," explained Frances S. Hoch, the North Carolina education department's chief consultant on second-language studies and a member of the Spanish advisory committee for the revamped tests. The ETS is hoping the new line of testing will allay such concerns by offering more information about prospective teachers.
The challenge is to redefine the concept of competence, said Sharon Robinson, director of the National Education Association's National Center for Innovation.3
One of the more radical ideas being floated, said Ms. Robinson, who served as a visiting fellow on the ETS project, is that test takers be asked to pose new ideas or questions, rather than merely supply correct answers to standardized queries.
"That would be a powerful assessment. I don't think the world is ready for that, though," she said, explaining that such fare might not be easily reduced to a score. "We'll have to come up with something that has to walk in both worlds."
Unlike the NTE, the new package of assessments will be administered during three distinct phases of professional training.
The testing service essentially started from scratch to produce the new tests. Although each stage will assess different characteristics, the ETS used a similar process to build the three phases. It reviewed the relevant educational literature and assorted state requirements, then sought the counsel of educators via telephone and in-person interviews.
Next, advisory committees were formed. Made up of teachers, principals, teacher educators, and representatives from state education departments and education groups such as the American Association of Colleges for Teacher Education, the committees also included more public-oriented organizations such as the National PTA.
As the final step, the ETS plans to survey thousands in the education community to balance the experts' views.
The overriding element in each stage has been to define what is important for a teacher to know or be able to do. Agreement has been "very strong" across all groups, ac cording to Michael Rosenfeld, director of professional and occupational studies for the testing firm. "Relatively new teachers are saying the same things are important as experienced teachers," he said.
The central questions posed at each stage of the assessment are:3
Stage I: Does a candidate have the basic skills to teach
Stage II: Does the individual know the subject matter and specific ways to teach it?
Stage III: Can the person apply this knowledge?
Stage I was designed to determine whether an individual has the basic reading, writing, and mathematics skills to become a teacher.4
Administered during the sophomore year in college, the test will en able students to realize early on if they have the skills required. As a result, Mr. Rosenfeld said, the process is "fairer to the student and it's fairer to the state and to the students in the classroom."
Instructional materials, broken down into distinct modules such as reading inference, will help students prepare for the test. Also included in the materials will be a practice test. The structure will permit students who fail the practice test or the actual examination to zero in on specific areas for further study.
In the computerized test, examinees will be asked to supply their own answers as well as select from multiple-choice responses. And they will be asked to write essays.
"It will be a multiple-length test depending on how well you answer the questions," said Paul A. Ramsey, Stage I leader for the ETS
A representative set of questions will be asked of all examinees. Once that set is completed, the computer will stop asking questions of individuals with scores high enough to have met state requirements. Test takers with failing scores will be given another set of questions for verification.
Despite the technology employed, computer skills will be unnecessary to succeed on the test, developers of the assessment said. Before the test, computerized tutorials will explain the symbols and process used.
What has surprised the ETS, officials interviewed here said, was the ease with which the test takers have used the computer. After students answer a question, for instance, the computer asks them to confirm their answer. "They see it as an [unnecessary] extra keystroke," said Kathleen O'Neill, a senior examiner.
"That was something we didn't think would happen. We expected more anxiety on their part."
The second part of the test will be administered at the end of under graduate training, when a candidate's knowledge of subject matter and pedagogy will be measured. Depending on state requirements, a test of content-specific pedagogy may also be required.
The Stage II test will consist of multiple-choice and "constructed response" questions calling for answers ranging from a single word to an essay.
A prospective English-literature teacher might be asked to do the fol lowing:
To test subject-matter knowledge, identify the author of a pas sage describing the mood and visionary qualities of a city.To test analytical ability, write an essay about the symbolism or social implications found in the works of Charles Dickens.To test classroom performance, prepare a lesson on A Tale of Two Cities.Test construction will be unique to each discipline, the test makers said. In addition to answering multiple-choice and constructed-response questions, for example, a prospective music teacher will be asked to submit a tape.
Douglass Fiero, the Stage II lead er, said he initially thought the tape was a frill, but that music teachers on the advisory committee convinced him that "if you can't make music in a classroom, you're not going to be a good teacher." The ETS is also attempting to take individual state requirements into consideration by developing modules for each discipline that will give states a menu of options from which to choose.
As the assessment is now conceived, teacher candidates will take a two-hour core test in their discipline in the morning. Optional tests will be given in the afternoon in one-hour chunks, for a maximum of four pieces.
This will account for the fact that one state may require only a mathematics examination, while another may mandate calculus or analytical algebra.
All told, no one will take more than six hours of tests in a single day.
Such tests will require more of the student and quite possibly the university, those involved in the project point out.3
For social studies, a teaching domain comprising a number of academic disciplines, the Stage II core test might consist of two 30-minute essays that would integrate U.S. history, world history, economics, geography, and civics/government. "You've got some disjuncture between what you teach and what you learn," said George L. Mehaffy, director of the education school at San Diego State University and a member of the social-studies advisory committee. "Candidates will have to know what the test contains and act accordingly to their [course] selection."3
"The astute candidate knows going down the pike he will have to take a geography course and an economics course," he said.
Mr. Mehaffy also theorized that the new tests could change the way liberal-arts courses are taught. If, for instance, a significant number of biology majors have difficulty passing the content portion of the teacher exam, he said, "we're going to have to look not only at the candidates, but to the preparation they received."
Stage III of the new assessment will not occur until the teacher candidate is in the classroom. Least developed of the three stages because it requires field work in partnership with the states, this performance-based assessment is wholly new to the ETS The third phase is "the trickiest and least well known," said Andrew C. Porter, director of the Wisconsin Center for Education Research at the University of Wisconsin-Madison and a member of the ETS teach programs council. But it has the "potential to be a great contribution," he said.
Stage III will consist of a series of observations and evaluations conducted by state and local school per. A candidate will also be expected to provide work samples and possibly submit to additional writ ten questions or interviews."
The focus, ETS officials said, will be on such skills as lesson planning, instruction, classroom management, student evaluation, and instructional effectiveness.
Experts on teacher assessment have generally welcomed the testing firm's efforts on the overhaul project.
"They're on the right track," said Raymond L. Pecheone, chief of the Bureau of Research, Evaluation, and Teacher Assessment for the Connecticut Department of Education. He said the integrity of the new testing instruments, however, will not be validated until the scoring components are pilot tested.
"The project is well-conceived," added Ms. Robinson, citing the incorporation of open-ended responses and performance-based assessments.
"There are some issues that remain on the table," Mr. Porter said. At preH sent, the University of Wisconsin re searcher observed, the focus is on the beginning teacher, but the test may be appropriate for more experienced teachers as well.
That issue is a potentially sensitive one in light of the efforts of the National Board for Professional Teaching Standards to offer certification to seasoned professionals. ETS officials describe the relationship between their new examination and the national board's plan as complementary rather than competitive.
Other issues that need to be re solved, Ms. Robinson suggested, are whether the test can be offered at a reasonable price, whether the technology will be accessible, and whether teachers will be presented with evidence that "it is an important rite of passage rather than a useless political barrier."