Some said it would devastate working relationships between teachers. It hasn’t. Others hoped it would dramatically raise student performance. Not so far.
But Denver’s closely watched experiment in performance-related pay for teachers nonetheless is leaving a big mark on the city’s school system, as described in an extensive report released last week on the 2-year-old pilot project.
Although many questions remain unanswered, the effort is pointing up the difficulty of putting such a compensation system into place, while at the same time suggesting that the goals teachers set for themselves may play an important role in their students’ success.
Read the report, “Pathway to Results: Pay for Performance in Denver.” (Requires Adobe’s Acrobat Reader.)
“There really are some major learnings to be seen here,” said William J. Slotnik, the executive director of the Community Training and Assistance Center, the nonprofit, Boston-based consulting group that drafted the report as part of an ongoing study of the initiative. “But I don’t think we are at a point where we can say that pay for performance is the way to go or that it’s not the way to go.”
Easier Said Than Done
Denver’s unusual project has promised to add a missing piece to the puzzle over how to move beyond traditional teacher-compensation systems, which base salary on education levels and years of service. Elsewhere, most new pay plans reward educators either for schoolwide improvement on student tests or for showing through reviews that teachers have acquired new skills.
But in the Mile High City, local union and district leaders agreed to build a link between what individual teachers are paid and the academic progress students in their classrooms make. At the 16 participating schools, each teacher works with administrators in the fall to draft a pair of improvement objectives. Meeting the two goals garners a bonus of $1,500. Many teachers, for instance, pledged that specific numbers of their students would score at the proficient level on certain tests by the end of the year.
That may sound simple enough, but the pilot hasn’t unfolded exactly as planned. For instance, organizers initially envisioned the effort as a comparison of three different ways of setting teacher objectives. But in practice, the goals set by teachers didn’t break down neatly into three discernible types, so the experiment’s focus has shifted more toward comparing the progress of pilot schools against that of other schools in the 74,000-student district.
A major difficulty relates to the assessments available. Colorado’s state exam system isn’t yet set up to allow for tracking individual students’ progress from year to year. And while Denver also uses the Iowa Tests of Basic Skills, the commercially available assessment isn’t aligned with local or state curriculum standards. Plus, some subjects, such as music and American government, aren’t covered by either exam. Teachers have made do with what they have.
“Denver now is having to grapple with some of the issues affecting every district in the United States,” Mr. Slotnik said, “which is that the assessments we use are not necessarily designed for these new purposes.”
The pilot also has stretched the district’s ability to maintain information on student performance. One lauded byproduct of the experiment is a new intranet system the district launched so that teachers can now review extensive computerized data on their students’ academic histories. But Mr. Slotnik says more needs to be done to coordinate the operations of Denver’s assessment, curriculum, and personnel systems.
Objective Results
Despite such challenges, the 150-page report suggests that the worst fears about the pilot have not been borne out. A survey of teachers involved in the experiment last year showed 87 percent agreeing that cooperation among staff members had either stayed the same or improved following the initiative.
“I think it flies in the face of the prejudice that says teachers would resist this,” said Brad Jupp, who has been on leave from his job as a Denver teacher to help design the pay plan. Mr. Jupp’s union, the Denver Classroom Teachers Association, is an affiliate of the National Education Association, which opposes linking pay to test scores.
Harder to glean so far is evidence on how schools’ participation in the program may be affecting student achievement. Results from the first two years appear to show most pilot schools outpacing their counterparts on the state’s assessments in reading, but failing to keep up on the Iowa tests. Few of the differences were statistically significant, however.
In the meantime, a clearer picture is starting to emerge about the importance of the teachers’ objectives. Reviewers with the Community Training and Assistance Center rated educators’ written goals on a 4-point scale, based on such measures as how well they explained their rationales and how clearly they spelled out their expectations for students. Teachers whose objectives earned the highest scores were also the ones who tended to see the greatest gains in their students’ performance.
In fact, that finding held true whether or not the teachers actually met their objectives and won bonuses. Mr. Slotnik says it makes sense.
“A lot of what goes into creating a high-quality objective,” he said, “are the same things that go into creating a high-quality lesson plan.”
Denver officials plan to continue the pilot through the 2002-03 school year, during which more research will be carried out to gauge the program’s effects. The expectation among district and union leaders is that lessons from the bonus program will be used in crafting a proposal for a wider-reaching system of performance-based pay for the whole district.
“My personal feeling is that, as of yet, it has not had the kind of impact it’s got to have if we are going to have it be adopted and be effective,” said Lester Woodward, a member of the Denver school board. “And that’s really a big part of the challenge over the next two years.”
