School & District Management

‘Effective Teaching’ Study Seen as Influential, and Faulty

By Stephen Sawchuk — November 06, 2013 4 min read
  • Save to favorites
  • Print

Of all the grants the Bill & Melinda Gates Foundation has made in teacher quality, observers tend to agree that the single most influential has been the $45 million Measures of Effective Teaching study.

Nearly all the researchers interviewed about the study praised its technical merits. But that hasn’t silenced the criticism aimed at how the project was framed, how the findings were communicated, and whether the many states drawing on them to draft teacher-evaluation policies are doing so appropriately.

The study’s core findings are that “value added” models, observations of teachers keyed to frameworks, and student survey results all, to an extent, predict which teachers help their students learn more. Combined into a single measure, they offer trade-offs of validity, stability, and cost.

One thread of criticism: The research made students’ test scores paramount, said Bruce D. Baker, a professor in the graduate school of education at Rutgers University in New Brunswick, N.J.

The Gates Foundation and the research team set up “a research framework that really boxed them in,” he said. “Throughout the course of these studies, it was always assumed that the validity check for anything and everything else was next year’s value-added scores.”

Contribution to Learning

Value-added models, which try to determine teachers’ contributions to student learning, are supposed to make the use of test scores fairer to teachers by taking into account students’ performance history and backgrounds. But teachers remain deeply skeptical of those models.

The summaries of the teaching-effectiveness research released for the public beginning in 2011, meanwhile, emphasized certain conclusions and downplayed other important findings, critics charge.

Education Week Receives Gates Aid

The Gates Foundation has provided grant support to Editorial Projects in Education, the nonprofit corporation that publishes Education Week. The newspaper retains sole editorial control over coverage. See disclosure.

For one, said Jay P. Greene, a professor of education reform at the University of Arkansas at Fayetteville, the reports stressed the importance of observation tools despite their cost and generally weak correlation with teachers’ future performance. Similarly, one less prominently featured finding was that a teacher-observation framework in English/language arts called PLATO actually did better than value-added in some cases in predicting how a teacher’s students would perform on higher-order, more cognitively challenging tasks in that field.

“It’s difficult to write data up when they’re controversial and you’re not sure what to emphasize,” said Susanna Loeb, a professor of education at Stanford University who was on the project’s technical-advisory committee but didn’t conduct any of the research. “I think there are a lot of interpretations about what the results mean. And the study doesn’t tell you the effect of using any of these measures in teacher evaluation in practice.”

Debates about the study tend to reflect disagreements about the translation of the research into policy. Jesse Rothstein, a University of California, Berkeley associate professor of public policy, for instance, described the relationship between various ways of estimating the value-added measures as “shockingly weak,” calling into question their usefulness as a factor in personnel decisions.

State Capacity

But the principal researcher on the study, Thomas Kane of Harvard University, disputes such characterizations. He argues that the correct basis for comparing the strength of the study’s identified measures is the information districts have traditionally used instead.

“It’s not like we can avoid making high-stakes decisions about teachers,” he said. “The right comparison is not to perfection; it’s to experience and master’s degrees and the information we currently have. Relative to that information, do these measures do better? The answer is unequivocally ‘yes.' "

Lawmakers and state education officials digesting the study’s results, meanwhile, are bumping up against the fact that not all states have the capacity to generate value-added data. Many states, such as New Jersey, are using an alternate method of gauging teachers’ impact on test scores, called student-growth percentiles, that the research didn’t even undertake, sparking concern from scholars like Mr. Baker.

Bill Gates himself has harbored concerns about how some states and districts are putting into practice the ideas his philanthropy has catalyzed. In op-eds in national newspapers, he has opposed the publication of teachers’ evaluation results and the haste to establish tests in every subject to produce teacher-evaluation data.

Melinda Gates added in a recent interview with Education Week that state officials sometimes rushed to institute systems ahead of the teacher-effectiveness findings.

“When we come out with new research and new data, we can’t necessarily control how it spreads, nor should we,” she said. “It would have been nice and neat and tidy if we could have said, ‘Wait until the very last day when [the research] comes out, and this is the way to go.’ But I think some states went a little fast.”


EDUCATION WEEK RECEIVES GATES AID

Since 2005, the Bill & Melinda Gates Foundation has awarded more than $7 million in grants to Editorial Projects in Education, the nonprofit corporation that publishes Education Week.

That total includes a current grant of $2 million over 25 months to support the development of new content and services related to the education industry and innovation in K-12 education.

Also, the Gates Foundation provided a $2.6 million grant over 40 months, starting in 2009, to underwrite a range of efforts to support EPE’s editorial and business-development capacity.

The first grant from Gates to EPE, $2.5 million over four years, underwrote the Education Week Diplomas Count report, as well as original research on high school graduation rates, and related activities.

Education Week retains sole editorial control over the content of its coverage under the Gates grants.

Besides the grant support, the Gates Foundation in 2005 provided a $100,000 contract to EPE. Under the arrangement, the EPE Research Center conducted a pilot project on the feasibility of providing research support to the foundation.

A version of this article appeared in the November 07, 2013 edition of Education Week as ‘Effective Teaching’ Study Seen as Influential

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
IT Infrastructure & Management Webinar
From Chaos to Clarity: How to Master EdTech Management and Future-Proof Your Evaluation Processes
The road to a thriving educational technology environment is paved with planning, collaboration, and effective evaluation.
Content provided by Instructure
Special Education Live Online Discussion A Seat at the Table - Special Education: Proven Interventions for Academic Success
Special education should be a launchpad, not a label. Join the conversation on how schools can better support ALL students.
Special Education K-12 Essentials Forum Innovative Approaches to Special Education
Join this free virtual event to explore innovations in the evolving landscape of special education.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management Photos Six Years After Parkland Tragedy, Crews Demolish a Painful Reminder
The school building in Florida where a gunman killed 17 people is being pulled down. Victims' families have toured the site with lawmakers to push for change.
4 min read
Students, teachers, victims' families and passersby watch, Friday, June 14, 2024, as crews start the demolition of the Marjory Stoneman Douglas High School building where 17 people died in the 2018 mass shooting in Parkland, Fla. Officials plan to complete the weeks-long project before the school's 3,300 students return in August from summer vacation.
Students, teachers, and victims' families are among those watching on June 14, 2024, as crews start the demolition of the Marjory Stoneman Douglas High School building in Parkland, Fla., where 17 people died in the 2018 mass shooting. Officials plan to complete the weeks-long project before students return from summer vacation.
Wilfredo Lee/AP
School & District Management Download 'Science of Reading' Learning Walks: 4 Things for Principals to Look For
An instructional guide for school leaders to help implement shifts in reading practices.
1 min read
Photograph of a Black male teacher in the classroom with clipboard observing elementary students.
E+
School & District Management Opinion 4 Things School Leaders Should Do Before Setting Priorities
Sweeping language doesn't offer a road map for the school community. Here's why.
Peter DeWitt & Michael Nelson
4 min read
Screenshot 2024 06 12 at 7.16.56 AM
Canva
School & District Management As Districts Weigh 4-Day Weeks, Research Overlooks Their Most Pressing Questions
A new, searchable dashboard will help district leaders explore research on four-day school weeks.
4 min read
Illustration of people around a very large flip calendar with Mon-Thursday highlighted in red squares. The concept of task planning. People are engaged in planning a calendar schedule.
iStock/Getty