When a school district makes a big to-do about the use of “evidence to make decisions about how to help students learn, where to put our resources and how to manage our staff,” is it fair to criticize it for implementing unproven and experimental programs? skoolboy supports modest experimental innovations, as long as they are evaluated carefully before expansion to a scale that would encompass an entire population. After all, students and teachers aren’t guinea pigs. The fact that schools are failing is not a justification to do any old thing, on the assumption that any innovation will be better than the status quo.
Speaking of any old thing … The Washington Post reported earlier this week that the Washington, DC Public Schools are abandoning support for National Board certification as a means of teacher professional development, shifting instead to, among other things, the Skillful Teacher program marketed by Research for Better Teaching, Inc. (RBT), founded by Jon Saphier in 1979. The program consists of a series of six one-day workshops; you can buy the book, which approaches 600 pages, for $70 at Amazon.
You might think that an organization that’s been peddling professional development for 30 years, with a book in its sixth edition, would have some compelling evidence of the effects of the jewel in its crown on teaching practice and student learning. If professional development doesn’t result in improvements in teaching and learning, what’s the point? But the RBT website doesn’t point to much evidence, emphasizing testimonials and brief “stories.” skoolboy’s favorite is the account of Fairfax County, VA’s implementation, “Making Teacher Evaluation Substantive and Growth-Oriented.” “In the first year of implementation, 162 teachers were dismissed or resigned compared to single digits the previous year,” the website crows. Now, is that growth, or is it development? Sometimes skoolboy gets confused by the difference.
The blurb for Montgomery County, MD, on which the DC plan is based, touts an independent evaluation by Dr. Julia Koppich of the program’s effects on teachers and administrators, and claims that “in 2001 grade 2 students scored in the 68th percentile in math computation. In 2003 scores were in the 83rd percentile.” The inference is that the Skillful Teacher program produced this change, but any reader of this blog knows that demonstrating program impact requires a careful design to rule out alternative explanations of changes over time in outcomes. (It also helps to have a good theory of how a program might plausibly produce particular changes.)
Montgomery County’s own internal evaluations of Studying Skillful Teaching aren’t as positive. Although 3rd grade teachers and Alegebra I teachers who took the course are “more likely to teach mastery lessons and less likely to miss opportunities to positively impact student learning” than comparable teachers who did not (Merchlinsky, 2006, 2007), there were no effects on elementary reading and math test scores or algebra performance.
And lest anyone think that people who live in glass houses shouldn’t throw stones, The Skillful Teacher isn’t the only professional development initiative that skoolboy, who teaches at the home of the Teachers College Reading and Writing Project, thinks could benefit from more rigorous evaluations before scaling up.