School & District Management

Popular Frameworks Found to Identify Effective Teachers

By Stephen Sawchuk — January 06, 2012 5 min read

Teachers who embodied the teaching skills outlined in certain popular teaching frameworks tended to help their students learn more, concludes a new study released by the Bill & Melinda Gates Foundation.

The report is the second major release from the foundation’s Measures of Effective Teaching project, which seeks to identify the best gauges of effective teaching. Ultimately, the Seattle-based foundation plans to devise a prototype teacher-effectiveness measure based on the findings.

Among other implications, the study’s results suggest that observations of teaching practice hold promise for being integrated into teacher-evaluation systems—if observers are carefully trained to ensure consistent application of the frameworks over multiple observations. Also, the study indicates that the gauges that appear to make the most finely grained distinctions of teacher performance are those that incorporate many different types of information, not those that are exclusively based on test scores.

“I was surprised at how aligned all the measures were,” said Douglas O. Staiger, a professor of economics at Dartmouth College, in Hanover, N.H., and one of the principal researchers on the study. “They seem to pick out teachers who are good on a range of dimensions, and I think that says there really is something kind of coherent about good teaching.”

Other studies have also linked teaching frameworks to student growth. But the breadth of measures studied, the number of districts and teachers included, and the focus on reliability in the Gates Foundation analysis give a richer picture of how stakeholders might create evaluation systems based on multiple measures, said Douglas N. Harris, an associate professor of education policy and public affairs at the University of Wisconsin-Madison, who was not involved in the study.

Still, he added, “all of this is being done in a low-stakes context. The question going forward is to what extent these conclusions apply in a high-stakes setting.”

The Gates Foundation also provides grant support for Education Week’s coverage of the education industry and for organizational capacity-building by the newspaper’s nonprofit publisher.

Findings Rundown

The study, “Gathering Feedback for Teaching,” released last week, draws on some 7,500 videotaped lessons taught by more than 1,300 grade 4-8 teachers across six school districts in several states. Each lesson was scored by multiple observers trained on one of several teaching frameworks.

The two general frameworks studied are consultant Charlotte Danielson’s Framework for Teaching and the Classroom Assessment Scoring System, a rubric designed by Robert C. Pianta, currently the dean of the education school at the University of Virginia, in Charlottesville. Three subject-specific frameworks, one in English/language arts and two in mathematics, were also tested.

The scores were then analyzed by looking at a variety of outcomes for students taught by those teachers. Among the new findings:

• All five frameworks were shown to bear a positive relationship to student achievement, as measured by growth on both state tests and more cognitively challenging exams, though the correlation was often “modest” in size.

• The error associated with the instruments was quite large when based on one observation by one observer, but it decreased when multiple observers scored the lessons.

• The observation frameworks showed a positive correlation with nonacademic measures of student happiness and effort.

• Teachers tended to score higher on such aspects of teaching as classroom management and pacing, but lower on such skills as connecting lessons to other disciplines, employing problem-based approaches, and using questioning strategies.

The MET project’s first release, in December 2010, brought some complaints from teacher groups, which said that the baseline measure of student outcomes—growth in test scores—was too narrow. For this study, the researchers broadened the list of outcomes slightly to include a measure of student effort and emotional engagement. Students taught by the teachers studied reported, for instance, on whether they pushed themselves to understand lessons in the class, and whether they felt happy in class.

The study also found that the observations helped improve the precision and stability of teacher-effectiveness calculations when they were combined with the two measures from the previous study: students’ perceptions of their teachers and “value added” measures of student test-score growth, a controversial element in many states’ new teacher-evaluation systems.

For instance, the study estimated that the difference in learning between students taught by teachers in the top and bottom quartiles of effectiveness, using just the Framework for Teaching as a measure, amounted to about 2½ months of growth in learning. But when the additional factors were included, the spread between top and bottom teachers’ effectiveness grew to 7½ months of learning, or almost a full year of schooling.

The researchers stressed that any decision to unite such measures carries trade-offs. For instance, of the measures studied, value-added was generally the best single predictor of future teacher performance, as measured by student score gains on state tests, but it is not as stable from one year to the next as student perceptions.

“There are trade-offs to what you put weight on, especially if you have goals beyond just maximizing how teachers can have the biggest impact on state standardized tests,” Mr. Staiger said.

In addition to the report’s implications for teacher-quality policy, its findings raise new questions about the state of testing in English/language arts. Researchers have long noted that value-added measures of teacher effectiveness seem to be less responsive to instructional differences in that subject than in mathematics.

The Gates Foundation study, however, found that student achievement on the Stanford 9, a standardized, open-ended literacy test requiring students to explain their thinking in writing, was more sensitive to differences in teacher quality than the state English/language arts tests.

The Context of Teaching

The report’s authors hypothesize that most states’ tests emphasize reading comprehension over the important skill of writing, which is emphasized in later grades.

The findings are unlikely to end what has so far been a volatile national discussion about how to boost teachers’ skills.

The president of the American Federation of Teachers, Randi Weingarten, said the study shows that teacher evaluation needn’t be based primarily on test scores. But she added that the study still puts too little emphasis on the context of teachers’ work.

“They are still regulating teachers, rather than creating the steps, including asking them what they need, to improve their practice,” she said in an email.

Among the MET analyses still under way is a random-assignment study meant to determine whether students assigned to teachers identified as more effective actually learn more relative to their peers.

A version of this article appeared in the January 11, 2012 edition of Education Week as Study: Popular Teaching Frameworks Can Help in Identifying ‘Good Teachers’

Events

Student Well-Being Webinar Boosting Teacher and Student Motivation During the Pandemic: What It Takes
Join Alyson Klein and her expert guests for practical tips and discussion on how to keep students and teachers motivated as the pandemic drags on.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
A Holistic Approach to Social-Emotional Learning
Register to learn about the components and benefits of holistically implemented SEL.
Content provided by Committee for Children
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
How Principals Can Support Student Well-Being During COVID
Join this webinar for tips on how to support and prioritize student health and well-being during COVID.
Content provided by Unruly Studios

EdWeek Top School Jobs

Interdisciplinary STEAM Specialist
Smyrna, Georgia
St. Benedict's Episcopal School
Interdisciplinary STEAM Specialist
Smyrna, Georgia
St. Benedict's Episcopal School
Arizona School Data Analyst - (AZVA)
Arizona, United States
K12 Inc.
Software Engineer
Portland, OR, US
Northwest Evaluation Association

Read Next

School & District Management New York City's Equity-Minded Schools Chief Resigns
Richard A. Carranza, the chancellor of the New York City schools, announced Feb. 26 he will step down from the job next month.
4 min read
Richard Carranza, Chancellor of the New York City Department of Education, arrives to Public School 188 The Island School as students arrive for in-person classes, on, Sept. 29, 2020, in the Manhattan borough of New York.
Richard A. Carranza announced he will depart the top New York City schools job in March.
John Minchillo/AP
School & District Management Opinion New Resource Tracks School System Reopening
The Return to Learn Tracker identifies the current instructional model of all regular public school districts with three or more schools.
5 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
School & District Management San Francisco School Board Pauses Renaming 44 Schools, Promises to Consult Historians
The renaming of 44 schools in the San Francisco Unified School District is apparently being put on hold after intense blowback.
Greg Keraghosian
1 min read
A pedestrian walks below a sign for Dianne Feinstein Elementary School in San Francisco, on Dec. 17, 2020. The San Francisco Unified School District put the renaming of 44 schools, including Dianne Feinstein Elementary School, on hold after local and national blowback.
A pedestrian walks below a sign for Dianne Feinstein Elementary School in San Francisco, on Dec. 17, 2020. The San Francisco Unified School District put the renaming of 44 schools, including Dianne Feinstein Elementary School, on hold after local and national blowback.<br/><br/>
Jeff Chiu/AP
School & District Management Superintendent Who Led During COVID-19 School Shutdowns Gets Top Honors
Michelle Reid of Washington state's Northshore district, one of the very first to close schools last March, was named National Superintendent of the Year.
3 min read
Michelle Reid, superintendent of the Northshore district in Washington
Michelle Reid, the superintendent of the Northshore district in Washington, was named National Superintendent of the Year.
courtesy of AASA, the School Superintendents Association