School & District Management

Gates Analysis Offers Clues to Identification of Teacher Effectiveness

By Stephen Sawchuk — January 11, 2011 4 min read
  • Save to favorites
  • Print

“Value added” gauges based on growth in student test scores and students’ perceptions of their teachers both hold promise as components of a system for identifying and promoting teacher effectiveness, according to preliminary findings from the first year of a major study.

The analysis, released last month by the Bill & Melinda Gates Foundation, shows that teachers’ value-added histories were among the strongest predictors of how they would perform in other classrooms or school years—as were students’ perceptions of their teachers’ ability to maintain order and provide challenging lessons.

The findings are part of the Seattle-based foundation’s $45 million Measure of Effective Teaching study, which seeks to identify accurate measures of superior teaching. (“Multi-City Study Eyes Best Gauges of Good Teaching,” Sept. 2, 2009.)

While underscoring the preliminary nature of the findings, Gates officials said they were heartened to see that several measures being studied do appear predictive of good teaching.

“I was hugely excited and encouraged” by the findings, said Vicki Phillips, the foundation’s director of education programs. “It has implications for what people can be doing right now. It begins to answer questions teachers have had. And I think it shows that valid teacher feedback doesn’t need to be limited to test scores alone.”

Among its education philanthropy, the Gates Foundation provides grant support to Editorial Projects in Education, the publisher of Education Week.

Complex Studies

The preliminary findings are based on data from five of the six districts participating in the study. They are New York City; Charlotte-Mecklenburg, N.C.; Hillsborough County, Fla; Dallas; and Denver.

A team of researchers directed by Thomas J. Kane, the foundation’s deputy director of education research and data, analyzed student scores on state tests given in grades 4-8 in the 2009-10 school year, using value-added modeling.

Such modeling purports to control for a student’s past performance and other factors so that learning gains can be attributed to specific teachers.

The researchers also analyzed student-perception data gathered from 2,519 classrooms, grades 4-8. Students rated teachers on a 1-to-5 scale on such aspects as whether teachers made the point of their lessons clear, were considerate of students, and explained material in several different ways.

The analysts found that, in every grade and subject studied, teachers’ value-added histories were strongly predictive of their performance in other classrooms. While such estimates exhibit a degree of volatility in the estimates from year to year, that volatility “is not so large as to undercut the usefulness of value-added as an indicator of future performance,” the study says.

Similarly, the researchers found that student perceptions of a given teacher were generally consistent across his or her classes, and that students gave high ratings to teachers whose classes consistently made learning gains.

In particular, student perceptions of teachers’ ability to manage a classroom and provide challenging academic content were strongly linked to those teachers’ ability to raise scores.

One of the study’s findings appears to complicate the conventional wisdom that teachers can boost scores by “teaching to the test.”

The analysis found that the value-added estimates of teacher effectiveness held up even when students were given supplemental tests with harder tasks less subject to test preparation than those on the state tests, including conceptual questions and open-ended writing tasks.

Meanwhile, student reports of classes spent heavily on test preparation were generally weaker predictors of teachers’ ability to raise scores than other factors, though the study did find a positive relationship between test preparation and teacher value-added estimates.

Combining Results

The value-added findings, in particular, come in the midst of a divisive debate in the K-12 field about whether such methods should count in a teacher’s evaluation.

The Gates Foundation’s findings on student perceptions, in the meantime, raise new questions for states and districts. Spurred by federal grant programs, some states have moved toward including teacher observations and even value-added methods in evaluations. Far fewer, however, are considering student-perception data.

So far, the study also appears to support the notion, advocated by teachers’ unions and others, that evaluations should be based on multiple measures. The analysis concludes that combining both sources of information—value-added and student feedback—yielded a more finely grained estimate of teacher effectiveness than using the student-perception information alone.

One key area not yet studied is the accuracy of teacher-observation ratings on a variety of teaching frameworks. The foundation’s research partners are still collecting and scoring videotaped observations of some 13,000 lessons in 2009-10 as part of that effort.

Other measures under study include teachers’ pedagogical content knowledge and their perceptions of their working conditions.

Gates officials plan to release a second report next spring. It will begin to examine the results of an experiment, already under way, to gauge student performance when students are randomly assigned to teachers identified as being more or less effective. Final results will be released in the winter of 2011-12.

“Things we’ve intuitively known, or thought about, or wished for about teacher effectiveness—there’s now some empirical evidence that they are valid,” Ms. Phillips said.

A version of this article appeared in the January 12, 2011 edition of Education Week as Gates Analysis Offers Clues to Identification of Teacher Effectiveness

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
Start Strong With Solid SEL Implementation: Success Strategies for the New School Year
Join Satchel Pulse to learn why implementing a solid SEL program at the beginning of the year will deliver maximum impact to your students.
Content provided by Satchel Pulse
Teaching Live Online Discussion Seat at the Table: How Can We Help Students Feel Connected to School?
Get strategies for your struggles with student engagement. Bring questions for our expert panel. Help students recover the joy of learning.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Science Webinar
Real-World Problem Solving: How Invention Education Drives Student Learning
Hear from student inventors and K-12 teachers about how invention education enhances learning, opens minds, and preps students for the future.
Content provided by The Lemelson Foundation

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management Why Principals Must Focus on SEL, School Climate Right Now
Principals need support in focusing on SEL, academic development, school climate, and community engagement, a new report argues.
6 min read
Image of a leader reflecting on past and future.
akindo/DigitalVision Vectors
School & District Management School Districts Showcase What's Working to Improve Student Learning
School leaders from 13 districts shared strategies at a national summit by AASA, the School Superintendents Association.
3 min read
David Schuler, superintendent of High School District 214 near Chicago, Ill., speaks about college and career readiness during a presentation at AASA's first annual Learning 2025 Summit on Tuesday, June 28, 2022, in Washington, D.C. High School District 214 is one of 13 "lighthouse" districts that were recognized for innovative work to improve school systems.
David Schuler, superintendent of High School District 214 near Chicago, speaks about college and career readiness at a summit in Washington.
Libby Stanford/Education Week
School & District Management Schools Prefer Cheaper Ventilation Options to Curb COVID: Why They Should Consider Upgrading
Most schools are opening windows and hosting class outdoors rather than investing in costlier, more-effective measures.
2 min read
Students from PS 11 Elementary School participate in art projects and interactive activities, during an after-school outdoor program held in the High Line park in New York, NY, October 21, 2020.
Students from PS 11 Elementary School participate in art projects and interactive activities during an after-school outdoor program in New York City in 2020. Many schools are opting for outdoor classes and other-low cost measures to maintain healthy air quality during the pandemic.
Anthony Behar/Sipa via AP Images
School & District Management Interactive Hour by Busy Hour: What a Principal's Day Actually Looks Like
From the time they wake up until they set the alarm at night, school leaders juggle the routine, the unexpected, and the downright bizarre.
Left, Principal Michael C. Brown talks on a radio at Winters Mill High School in Westminster, Md., on May 17, 2022. Right, Boone Elementary School principal Manuela Haberer directs students and parents in the pick-up line at the conclusion of the school day on May 19, 2022 in San Antonio, Texas.
Left, Principal Michael C. Brown talks on a radio at Winters Mill High School in Westminster, Md., on May 17, 2022. Right, Boone Elementary School principal Manuela Haberer directs students and parents in the pick-up line at the conclusion of the school day on May 19, 2022 in San Antonio, Texas.
From left, Steve Ruark and Lisa Krantz for Education Week