School & District Management

Popular Frameworks Found to Identify Effective Teachers

By Stephen Sawchuk — January 06, 2012 5 min read
  • Save to favorites
  • Print

Teachers who embodied the teaching skills outlined in certain popular teaching frameworks tended to help their students learn more, concludes a new study released by the Bill & Melinda Gates Foundation.

The report is the second major release from the foundation’s Measures of Effective Teaching project, which seeks to identify the best gauges of effective teaching. Ultimately, the Seattle-based foundation plans to devise a prototype teacher-effectiveness measure based on the findings.

Among other implications, the study’s results suggest that observations of teaching practice hold promise for being integrated into teacher-evaluation systems—if observers are carefully trained to ensure consistent application of the frameworks over multiple observations. Also, the study indicates that the gauges that appear to make the most finely grained distinctions of teacher performance are those that incorporate many different types of information, not those that are exclusively based on test scores.

“I was surprised at how aligned all the measures were,” said Douglas O. Staiger, a professor of economics at Dartmouth College, in Hanover, N.H., and one of the principal researchers on the study. “They seem to pick out teachers who are good on a range of dimensions, and I think that says there really is something kind of coherent about good teaching.”

Other studies have also linked teaching frameworks to student growth. But the breadth of measures studied, the number of districts and teachers included, and the focus on reliability in the Gates Foundation analysis give a richer picture of how stakeholders might create evaluation systems based on multiple measures, said Douglas N. Harris, an associate professor of education policy and public affairs at the University of Wisconsin-Madison, who was not involved in the study.

Still, he added, “all of this is being done in a low-stakes context. The question going forward is to what extent these conclusions apply in a high-stakes setting.”

The Gates Foundation also provides grant support for Education Week’s coverage of the education industry and for organizational capacity-building by the newspaper’s nonprofit publisher.

Findings Rundown

The study, “Gathering Feedback for Teaching,” released last week, draws on some 7,500 videotaped lessons taught by more than 1,300 grade 4-8 teachers across six school districts in several states. Each lesson was scored by multiple observers trained on one of several teaching frameworks.

The two general frameworks studied are consultant Charlotte Danielson’s Framework for Teaching and the Classroom Assessment Scoring System, a rubric designed by Robert C. Pianta, currently the dean of the education school at the University of Virginia, in Charlottesville. Three subject-specific frameworks, one in English/language arts and two in mathematics, were also tested.

The scores were then analyzed by looking at a variety of outcomes for students taught by those teachers. Among the new findings:

• All five frameworks were shown to bear a positive relationship to student achievement, as measured by growth on both state tests and more cognitively challenging exams, though the correlation was often “modest” in size.

• The error associated with the instruments was quite large when based on one observation by one observer, but it decreased when multiple observers scored the lessons.

• The observation frameworks showed a positive correlation with nonacademic measures of student happiness and effort.

• Teachers tended to score higher on such aspects of teaching as classroom management and pacing, but lower on such skills as connecting lessons to other disciplines, employing problem-based approaches, and using questioning strategies.

The MET project’s first release, in December 2010, brought some complaints from teacher groups, which said that the baseline measure of student outcomes—growth in test scores—was too narrow. For this study, the researchers broadened the list of outcomes slightly to include a measure of student effort and emotional engagement. Students taught by the teachers studied reported, for instance, on whether they pushed themselves to understand lessons in the class, and whether they felt happy in class.

The study also found that the observations helped improve the precision and stability of teacher-effectiveness calculations when they were combined with the two measures from the previous study: students’ perceptions of their teachers and “value added” measures of student test-score growth, a controversial element in many states’ new teacher-evaluation systems.

For instance, the study estimated that the difference in learning between students taught by teachers in the top and bottom quartiles of effectiveness, using just the Framework for Teaching as a measure, amounted to about 2½ months of growth in learning. But when the additional factors were included, the spread between top and bottom teachers’ effectiveness grew to 7½ months of learning, or almost a full year of schooling.

The researchers stressed that any decision to unite such measures carries trade-offs. For instance, of the measures studied, value-added was generally the best single predictor of future teacher performance, as measured by student score gains on state tests, but it is not as stable from one year to the next as student perceptions.

“There are trade-offs to what you put weight on, especially if you have goals beyond just maximizing how teachers can have the biggest impact on state standardized tests,” Mr. Staiger said.

In addition to the report’s implications for teacher-quality policy, its findings raise new questions about the state of testing in English/language arts. Researchers have long noted that value-added measures of teacher effectiveness seem to be less responsive to instructional differences in that subject than in mathematics.

The Gates Foundation study, however, found that student achievement on the Stanford 9, a standardized, open-ended literacy test requiring students to explain their thinking in writing, was more sensitive to differences in teacher quality than the state English/language arts tests.

The Context of Teaching

The report’s authors hypothesize that most states’ tests emphasize reading comprehension over the important skill of writing, which is emphasized in later grades.

The findings are unlikely to end what has so far been a volatile national discussion about how to boost teachers’ skills.

The president of the American Federation of Teachers, Randi Weingarten, said the study shows that teacher evaluation needn’t be based primarily on test scores. But she added that the study still puts too little emphasis on the context of teachers’ work.

“They are still regulating teachers, rather than creating the steps, including asking them what they need, to improve their practice,” she said in an email.

Among the MET analyses still under way is a random-assignment study meant to determine whether students assigned to teachers identified as more effective actually learn more relative to their peers.

A version of this article appeared in the January 11, 2012 edition of Education Week as Study: Popular Teaching Frameworks Can Help in Identifying ‘Good Teachers’

Events

Recruitment & Retention Live Online Discussion A Seat at the Table: Chronic Teacher Shortage: Where Do We Go From Here?  
Join Peter DeWitt, Michael Fullan, and guests for expert insights into finding solutions for the teacher shortage.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Science Webinar
Close the Gender Gap: Getting Girls Excited about STEM
Join female STEM leaders as they discuss the importance of early cheerleaders, real life role models, and female networks of support.
Content provided by Logitech
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Achievement Webinar
Mission Possible: Saving Time While Improving Student Outcomes
Learn how district leaders are maximizing instructional time and finding the best resources for student success through their MTSS framework.
Content provided by Panorama Education

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management Parents' Bad Behavior at School Sports Events Has Gotten Extreme
A adult brawl with deadly consequences at a boys' basketball game has renewed concern.
4 min read
Photographic negative image of aggressive spectators shouting at each others while others intervene.
iStock/Getty Images + EdWeek
School & District Management Opinion Your School Leadership Needs More Student Voice
When one Virginia principal moved from middle school to high school, he knew he would need to find new ways of soliciting student feedback.
S. Kambar Khoshaba
3 min read
Illustration of students holding speech bubbles.
Vanessa Solis/Education Week via Canva
School & District Management First Latina Selected to Lead National Principals Group
Raquel Martinez is a middle school principal in Pasco, Wash.
3 min read
Raquel Martinez, the principal of Stevens Middle School, in Pasco, Wash., was named president-elect of the National Association of Secondary School Principals. She’s the first Latina to hold the position.
Raquel Martinez, the principal of Stevens Middle School, in Pasco, Wash., was named president-elect of the National Association of Secondary School Principals. She’s the first Latina to hold the position.
Courtesy of the National Association of Secondary School Principals
School & District Management Four Things to Know From a State's Push to Switch Schools to Heat Pumps
Installing a heat pump is complex, but the payoff is well worth it, says an expert in Maine who's pushing their adoption in schools.
4 min read
Close up of a heat pump against a brick wall
E+/Getty