Both sides in the debate over whether state tests help or harm the quality of classroom teaching can draw support from a study presented here last week.
William A. Firestone and his colleagues at Rutgers University presented their findings in a series of papers here at the annual meeting of the American Educational Research Association. The data come from the first two phases of a three-year study, which surveyed more than 300 4th grade teachers in New Jersey about the effects of that state’s testing program on mathematics and science instruction. The researchers also observed 63 teachers’ classrooms.
The study found that the New Jersey Elementary School Performance Assessment, which includes a mix of multiple-choice and short-answer items, had encouraged teachers to try out more inquiry-oriented instruction in their classrooms. Such instruction included placing a greater emphasis on problem-solving, having students explain their thought processes, assigning students more writing, and making greater use of hands-on materials.
|
Rather than “dumbing down” teaching, the researchers suggest, the test “appears to be sensitizing teachers to new instructional practices even if it is not leading to major change.”
“The good news is that at a time when many critics of state assessments assume that teaching to the test means almost always driving instruction to the lowest level, we present a case of a state test encouraging teachers to consider more challenging approaches,” the researchers write. “The bad news is that a state test alone is not sufficient to help teachers make deep changes in their practice.”
At the same time, the study found that the tests could be encouraging more conventional and drill-oriented instruction in the state’s poor urban districts, where such teaching is already more prevalent.
New Jersey introduced its 4th grade math test on a statewide basis in 1998 and its science test in 1999. Although test results are reported publicly, they are not used to punish or reward individual schools or to determine whether students are promoted to the next grade.
Different Strategies
To reach their conclusions, the researchers collected data from a representative sample of 4th grade teachers in the state. In the spring of 1999, they conducted telephone surveys of 245 teachers and written surveys of 177 teachers, and collected portfolios of instructional materials from 110 teachers.
In the spring of last year, a total of 287 teachers, including 160 from the first year, responded to a telephone survey. In addition, the researchers observed the classrooms of 63 teachers, half of whom reported engaging in more inquiry-oriented instruction and half of whom stressed more conventional teaching techniques.
All of the teachers were asked how often they engaged in a range of test-taking strategies, from the use of “rubrics"— guides that set out criteria for evaluating student work—to teaching students such mechanics as filling in the bubbles on multiple-choice items. The researchers also asked teachers about the extent to which they engaged in more inquiry-oriented instruction, such as asking students to design their own science experiments, vs. more direct instruction, such as emphasizing the importance of following procedures to solve math problems.
The study found that different types of test preparation were associated with different forms of classroom instruction. “Teaching to the test may not always be the same thing,” Mr. Firestone, a professor of educational policy and theory, said while presenting his findings last week.
In particular, the researchers found, teachers who used rubrics in their classrooms were far more likely to engage in a range of instructional practices, such as having children argue about ideas or explain their solutions in writing, than were teachers who rarely used such rubrics in class. In contrast, teachers who reported almost always teaching the mechanics of how to take tests were far less likely to have children argue about ideas or to justify their conclusions in writing.
Lack of Depth
But while the tests generally were encouraging teachers to try out new instructional methods, the study found, those changes did not run deep. For example, manipulatives, such as blocks, were used in 73 of the 121 observed math lessons. But in the vast majority of cases, children were using the materials as directed by their teachers without any understanding of how the materials connected to the actual problem or to more symbolic representations of mathematics.
Similarly, 85 percent of the observed lessons focused only on procedural knowledge, without encouraging a deeper understanding of mathematical concepts.
“It appears that teachers are trying to make changes or thinking about making changes that they are not yet carrying off with substantial skill in the classroom,” the researchers note.
The study also found that while teachers were spending less time on basic arithmetic and more time on such new topics as statistics and problem-solving, there was little agreement about which new topics to emphasize. “If the American curriculum has been a mile wide and an inch deep in the past,” the researchers say, “there is a risk that New Jersey’s standards and assessments will amplify that trend.”
Similarly, the authors found that teachers reported spending more time on new science topics the first year the science test was introduced. But teachers cut back on science instruction after students scored relatively well on the exam, devoting more time to math and language arts, where students performed more poorly.
“Taken together, our findings suggest that state testing is neither the magic policy bullet that advocates of accountability hope for nor the force for deskilling, dumbing down, and disparity of life chances suggested by its opponents,” the researchers conclude. “Deeper changes in instructional practices and serious reduction in the inequities facing American children are likely to depend on orchestrating state tests with other policies and administrative practices.”
