In 2008, I wrote a feature article about a principal-training program in Pittsburgh that was wrapping up its first year. Called the Pittsburgh Urban Leadership System for Excellence, or PULSE, the initiative included a new evaluation system for principals, a leadership academy and private coaching for principals and central office staff members.
“We had to move on everything at once, because we weren’t doing any of it,” the superintendent at the time, Mark Roosevelt (Teddy’s great-grandson), told me.
And now four years after the program began (how time flies!) the RAND Corp. research group has come out with a report offering an assessment of the program’s effect on the 25,000-student school system. RAND is based in Santa Monica, Calif., but it has an office in Pittsburgh. The district plans to continue the work of PULSE, but RAND analyzed only the years that were funded by a federal Teacher Incentive Fund grant, which included the 2007-08 through 2010-11 school years.In addition to evaluating the program, RAND partnered with the district on the design of measures used to evaluate principal performance, and provided feedback and recommendations to the district on program improvement.
According to RAND, principals liked the program, which made them made them more active users of data for decision-making, and the program’s implementation coincided with student achievement increases for the district that exceeded the rest of the state.
A major component of PULSE is a opportunity for principals to increase their salaries by up to $2,000 based primarily on their performance on a rubric administered by assistant superintendents. The principals also could earn a yearly bonus of up to $10,000 based on student performance. But the study found that principals tended to link their own performance improvements to the support and feedback components of the initiative, rather than the financial incentives.
Principals gave high marks to the leadership academy created as part of the effort, saying it was particularly helpful in teaching them how to monitor staff members and in providing useful feedback. Two-thirds of the principals surveyed for the report said they appreciated one-on-one coaching from assistant superintendents.
In the area of student growth, test scores rose: By the fourth year of the program, the gains in grades 4 through 8 in math and reading reached their highest levels since the evaluation started. However, race and poverty achievement gaps also rose over the same time period. Researchers noted, however, that at the end of the four years of the study, low-achieving students started to see greater growth than they had earlier in the lifespan of the project.
The authors of the report say the work in Pittsburgh does not provide definitive evidence of the worth of performance bonuses for principals. However, it does offer ideas for how others might want to create evaluation systems for school leaders, as well as a system of professional development and targeted support, they wrote.
A version of this news article first appeared in the District Dossier blog.