I have been impressed by President-elect Obama’s pledge to steer clear of ideologically-driven policies, and instead choose to make policy based on the best ideas, regardless of their origin or political correctness. In that spirit, let’s take a look at the hot-button issue of assessment.
From former test-scorer Todd Farley comes a confession that the test scores by which our schools are judged are less than reliable. Mr. Farley worked scoring the National Assessment of Educational Progress, which calls itself nothing less than “The Nation’s Report Card.” He writes:
There’s not enough column space in this newspaper to list the myriad discrepancies I’ve seen in the scoring of short-answer/essay questions on “standardized” tests, but in my opinion, test scoring is akin to a scientific experiment in which everything is a variable. Everything. In my experience, the score given to every open-ended response, and ultimately the final results given to each student, depended as much on the vagaries of the testing industry as they did on the quality of student answers.
Mr. Farley goes on to describe in detail the limitations of his fellow scorers, some of whom had a limited grasp of the English language, and one who was unaware that he was actually scoring student work.
Of course you would never guess that there was any question about reliability from visiting the NAEP website. Nor would you guess from policy proposals that have emerged from the wreckage of NCLB that suggest that NAEP be used as the basis for some kind of national standardized test, allowing comparisons of students across the country. In fact, NAEP tests are so highly regarded that the No Child Left Behind Act requires states to participate in NAEP as a condition of receiving federal Title I funds.
I do not really know if NAEP is any better or worse than other standardized tests. I just hope that this revelation helps us look beyond the supposed precision provided by those ever-so-scientific looking test scores.
I also hope this leads us to take a broader view of assessment. Classroom-based assessments are often discounted as being unreliable and subjective. The role of the teacher has been reduced as standardized tests have become the crucial judges of our success. But when we read how the NAEP is being scored, we get the feeling that the vaunted objectivity of standardized tests may be less than it has been cracked up to be.
Research has shown that high school grades assigned by teachers are the best predictors of success in college, so maybe teachers are not as unreliable and subjective as we thought. Perhaps it is time to reinvest in teacher-based assessment practices. Teachers need to learn to assess more deeply, and apply what they are learning to provide students with timely feedback. It will take professional development and time to develop these skills. But if we took all the energy and resources that now go into high stakes tests and test preparation and turned that energy towards smarter authentic assessment practices, linked directly to classroom learning, I believe we would see better results, both in terms of the quality of assessment, and in terms of better-informed instruction.
If you aren’t sure what I mean by authentic assessment, take a look at some of the resources here on this site hosted by the University of Wisconsin-Stout, and this Authentic Assessment Toolbox created by Jon Mueller.
What do you think about the validity of high stakes tests such as the NAEP? How do you think our assessment practices should be strengthened?