Assessment Opinion

‘Garbage In, Garbage Out': Version 2

By Deborah Meier — March 21, 2013 5 min read
  • Save to favorites
  • Print

Editor’s note: Due to computer problems, the first version of Deborah Meier’s response to Eric Hanushek included several errors. This new version corrects the previous wording problems.

Dear Rick (vs. Eric, now that we know each other better!),

I’m going to try to go through your thoughtful response one step at a time, and then propose one of many alternatives.

First. Many of us have objected to standardized tests for students for decades (e.g. FairTest is hardly a young organization!). IQ tests and SATs have been the focus for half a century. My own writings on the subject began in the late 1960s. I luckily came across The Tyranny of Testing by physicist, Banesh Hoffman, at a time when I was struggling with test results on a personal level. Of course, when more and more stakes became involved--for kids as well as adults--alarm increased. Check my website bibliography for a look at the history of testing critiques.

Second. Yes, VAM (value-added measurement) is new, and, as you know, has been debunked for its unreliability and lack of validity by many. First of all, as they say, “Garbage in, garbage out.” No playing with refined statistics can make flawed data less so. If the first test score was invalid for making critical judgments, the second is too; and comparing them doubles their unreliability. The data I’ve seen suggests that the percentage of teachers who’d be fired one year and get a bonus the next is simply too high.

Third. Actually, the intention of common core is to test in all areas over time AND in the meantime to judge teachers on both, regardless of what they teach. My concerns we common core rest both on the CORE and above all on its assessment implications.

Fourth. A good system of observation might take note of various testing results of testing--including standardized ones. But in most states now the VAM trumps everything else no matter what the “contractual” percentage.

Fifth. Your concern that we fail to fire enough teachers is misleading. Too many folks do not acknowledge how many teachers leave because they are pushed out by so-called strong leaders, or they are counseled out by wise principals--in short, effectively fired--whatever it’s called. Often more for being “troublesome” (to the principal) rather than ineffective with their students and families. I’d guess that’s the No. 1 reason principals and superintendents harass teachers until they quit or transfer to another town. But otherwise, they are hardly alone in offering poor teachers the opportunity to quit vs being fired.

But to claim that teachers say “no way, not now, not ever” to evaluation ignores the fact that virtually all schools evaluate teachers annually. And have whether there’s a union or not. It’s done to and not with faculty. And pays a price for being top-down.

To do better at hiring and firing it helps to have a glut of teachers to draw from (“better a weak teacher than none " some principals argue) and if approached in a less adversarial way. More on this later. But no one I know has proposed a “scorched earth” approach--a strong word, Eric, when we’re trying to find common ground!

Sixth. The whole point of our complex judicial system is about “due process,” so why pass that off as a unworthy argument? It’s the cornerstone of our republic. Without assurances of due process, I don’t think I could have stayed in teaching--because I can be one of those “troublesome” people and I knew it.

Finally, the biggest lesson I learned over half a century is that the more democratically a school is run, the easier it is to “fire” weak teachers, especially truly “bad” ones. In a hierarchical school colleagues tend, virtually always, to come to the defense of a teacher who they see as being “pushed out” by the administration--usually without knowing who is right/wrong, and perhaps not knowing because there is no system for knowing that all agree on. It causes tension and morale problems.

Both CPESS (Central Park East Secondary School) and many other Coalition schools were set up with this problem in mind. (I’ll see if I can send you the Mission Hill manual on evaluation.) We use a committee of peers to regularly review/supervise/consult new colleagues. After a teacher is tenured we do this every few years. (The teacher being reviewed can suggest one member of the team.) The team drafts a recommendation, which is shared and responded to by the principal, and the teacher in question. The recommendation comes before the full faculty prior to our spring retreat so that both the school and the teacher have a chance to consider what to do next. After this the candidate can appeal to our school’s board (1/3 staff, 1/3 parents, 1/3 community members), and/or the superintendent and the union president. (It should be noted that the faculty council--on which I also sit does the hiring.)

Given this collegial approach I cannot imagine what benefits would come from adding teacher salaries into the mix. Thank god for the union--to which we all belong--for settling such questions. We do, however, also distribute an equal lump sum amongst all staff members in return for overtime and per-session moneys. (We figure averaging 5-10 extra hours a week on school matters.) It means we work hard but do not feel martyred. What’s harder to imagine is looking parents and kids in the eye and explaining why their teacher is ranked lower than her neighbor, or how we’d relate to each other if we were being annually ranked! When we have a new teacher we add resources: an extra adult or lower class sizes, as we do also when a teacher is going through a personal crisis (like divorce, death in the family, etc.).

We think we are modeling for students--openly and flagrantly--what we mean by a balance of powers, due process, the exercise of disciplined judgment, as well as the value of compassion. In retrospect these may be among the most powerful long-term lessons students learn from us.

Such schools depend on a climate of “pride” in each other’s work. We remind kids often that how they act reflects on the school’s reputation, which in turn will help or hurt them. We count on that from faculty as well. It changes the way we think about “our school.” 

We never fully live up to our mission statement, but we never lose sight of it.

This is more than you ever wanted to know.

The opinions expressed in Bridging Differences are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.