When it comes to writing essays, neatness may not always count. So concludes a study published in the fall issue of the Journal of Educational Measurement.
In their study, four researchers from the Educational Testing Service--Donald E. Powers, Mary E. Fowles, Marisa Farnum, and Paul Ramsey--sought to determine whether handwritten or word-processed essays produced higher scores on a pilot test for prospective teachers. They selected 32 students who were taking the test and asked them to produce two essays--one that was handwritten and one that was done on a computer.
After the essays were scored, professional word-processors converted the handwritten essays into word-processed versions but were instructed not to correct any spelling or grammatical errors. In the same way, professional transcribers converted the computer-written versions--errors and all--into handwritten copies. Trained readers who were unaware of the purpose of the study then re-scored all of the papers.
When the original handwritten essays were word-processed and re-scored, the average score decreased significantly. The scores of computer-written essays that were converted to handwriting increased slightly.
The researchers could not pinpoint why the neater-looking computer versions got lower scores. They speculated that one reason may be that the word-processed versions were shorter. They also noted that some of the handwritten versions showed that some of the examinees had revised their essays. Efforts at revision were not visible in the computer versions.
The researchers said their findings point to what could be a significant problem as more and more states and schools turn to performance-based measures of assessment to gauge students’ academic progress.
Educators at schools involved in reform efforts often say they value a more active, inquisitive brand of learning for their classrooms, but that message may not be getting across to students. That’s one of the findings of the Restructuring Collaborative, a loose organization of seven federal research laboratories working with schools engaged in reform.
Researchers, educators, and even students interviewed students at the schools involved in the collaborative to find out how they define a “successful learner.” To elementary students, they found, success means “pleasing the teacher.” Two-thirds of the middle school students said they thought school success hinges on compliance, making an effort, and getting good grades. And high school students defined it as “putting out the effort"--being responsible, having good study habits, being involved in school, and using well-developed organizational skills.
Robert E. Blum, the director of school, community, and professional development for the Northwest Regional Laboratory, which is helping spearhead the effort, says the researchers plan to feed the findings back to adults at those schools. “This should become part of what schools consider as they go about changing themselves,” he says. “A lot of restructuring efforts sort of miss the focus on learning and jump right in and focus on restructuring the schedule of the day, even though they may not be quite sure why.”
Even though new-style performance assessments seem to be generating controversy in some states and school districts, a new study suggests that parents prefer them to more traditional kinds of tests.
As part of a larger, ongoing study, researchers Lorrie A. Shepard and Carribeth L. Bliem of the University of California-based National Center for Research on Evaluation, Standards, and Student Testing showed the parents of 33 3rd graders sample multiple-choice questions from a standardized test and sample open-ended questions from a performance-based assessment.
By an overwhelming margin, parents said they favored the performance assessments because they “make children think” and allow teachers to better understand the child’s thinking process. But some of the parents also noted that they might have answered differently had they not had the opportunity to actually see the test questions.
The researchers said their findings suggest that simply exposing parents to the newer assessments may go a long way toward countering any misconceptions they have about them.
