That’s the big question taken up in a recently released report on the Teacher Incentive Fund grants. TIF is a federal initiative that couples performance pay for teachers and principals with other reforms in low-income schools.
The short answer after two years of implementation is yes, but it’s not a terribly large one. Interestingly, the effect shows up in reading, a subject in which test scores have proved harder to boost.
There’s also evidence that a lot of districts did not carry out the program the way it was intended, which might have affected these results. For instance, the bonus payouts were supposed to be “challenging to earn” for teachers. But according to the report, more than 60 percent of teachers got a bonus, suggesting they weren’t reserved for the highest-performing teachers.
The research, issued Sept. 24, was conducted on behalf of the U.S. Department of Education by the policy analysis firm Mathematica Policy Research.
TIF has been around since 2006. The report concerns the fiscal 2010 competition, when the feds set up an evaluation component. For one subset of grantees, some schools in the winning district were randomly assigned to the treatment group, in which they implemented all four of the program’s tenets (teacher evaluation, pay bonuses, a career ladder coupled with pay, and professional development). The others were assigned to a control group and told to do all of them except the bonus pay. The idea was to figure out just what impact the pay component had on student achievement.
As we reported about a year ago, the first year of study showed that a lot of the grantees struggled with communicating the admittedly complex program’s goals and features. Many teachers in the treatment schools were unaware that they were eligible to earn bonuses.
The analysis is based on surveys from 155 TIF districts, and interviews with and surveys of teachers and principals in the subset of 10 districts that participated in the random-assignment evaluation.
Year Two Findings
There’s a lot to parse in the second year of studies, but here are some of the main ones.
- Educators did understand the program better in year two: Of the teachers in the treatment schools, 62 percent reported being eligible for pay bonuses, up from just 49 percent in the first year of implementation. The principal figure jumped from 55 percent to more than 90 percent.
- Teachers in the treatment schools were less satisfied than those in the control group with the use of test scores to assess their performance. But they were happier with the opportunities to earn higher pay. (Just try to make sense of that one!)
- The pay component did not seem to boost teacher retention in treatment schools. But it did lead to more higher-performing principals staying on the job in those schools.
- Teachers in treatment schools got better effectiveness ratings when measured by student achievement, but not by ratings based on their classroom practice.
- Districts said that maintaining the program in the future posed major challenges.
The program did have effects on student achievement, in reading. This is notable, because most impact studies tend to show bigger effects in mathematics than reading. (That’s probably because reading is picked up outside school as well as in it, where nearly all math instruction happens in classroom.)
Here’s a graphic outlining the impact.
Putting the TIF Results In Context
Of course, a reasonable question to ask is: Does a one percentile-point gain percent matter much in terms of real learning?
Morgan Polikoff, a University of Southern California researcher, had this cheeky response.
@shermandorn new effect size scale: 1 point = meh, 3 = hmm, 5 = decent, 7 = wow, 10+ = BAM!
— Morgan Polikoff (@mpolikoff) September 24, 2015
Still, the findings stand in contrast to a 2010 random-assignment study that found that big pay boosts, by themselves, just didn’t seem to do much of anything. A smaller-scale study of Chicago’s TIF program, also from 2010, found no benefits for teacher-retention rates or for student achievement.
for the latest news on teacher policy and politics.
- Study Casts Cold Water on Bonus Pay
- Districts Faced Challenges Implementing Federal Performance-Pay Grants
- Performance-Pay Model Shows No Achievement Edge
A version of this news article first appeared in the Teacher Beat blog.