Find your next job fast at the Jan. 28 Virtual Career Fair. Register now.
Teaching Profession

Multi-City Study Eyes Best Gauges of Good Teaching

By Stephen Sawchuk — September 02, 2009 6 min read

The Bill & Melinda Gates Foundation will embark this fall on an ambitious research effort to analyze—and provide some initial answers to—a perennially vexing question in education: What are the best indicators of excellent teaching?

The foundation’s research partners intend to videotape and examine the teaching practices of 4,000 teachers in New York City, Charlotte-Mecklenburg, N.C., and several other yet-to-be named districts, to arrive at an understanding of the correlation between those practices and student learning.

In addition, the foundation will look at the relationship between student achievement and pupils’ perceptions of their instructors’ effectiveness; teachers’ content knowledge and ability to find the right pedagogical tools to teach that content; and “value added” estimates of teacher effectiveness based on test scores.

The research agenda makes up a component of the foundation’s five-year, $500 million push to define and promote effective teaching practices, and will help shape its approach to the teacher-effectiveness plans it intends to fund this fall. Foundation officials hope the research will help settle widely divergent opinions about what constitutes effective teaching—a lack of clarity that policy groups, administrators, and teachers’-union leaders alike say has contributed to poorly calibrated evaluation methods and perfunctory judgments that have left administrators frustrated and teachers bristling.

“One of the critiques teachers have—which we think is totally valid and fair—is that their performance shouldn’t be judged on one single measure of anything, whether test scores or observations,” said Vicki L. Phillips, the director of the foundation’s education division. “We want to take a very independent look at what indicators, when taken together, give you a fair, reliable, clear view of teacher effectiveness that both teachers and researchers can support and embrace.”

Among its other education philanthropy, the Seattle-based foundation provides grant support for Education Week.

‘Providing the Headlights’

The research project will occur alongside Gates’ “intensive partnerships for teacher effectiveness”—plans to retool recruitment, professional development, pay, and evaluation structures in selected districts. Four districts, and one consortium of charter schools, are finalists for that funding. The foundation will unveil its decisions in November. (“Finalists to Vie For Grants On Teaching,” Aug. 26, 2009.)

The foundation will invite teachers in some of those districts, in addition to New York and Charlotte, to participate in the research.The work in the selected sites will be done with the blessing of local leaders, educators, and teachers’ unions.

Gates officials could not provide an estimate of the overall amount they will spend on the research, but say the figure will be in the tens of millions of dollars. New York City will receive a $2.6 million grant to participate, while Charlotte is poised to receive $1.4 million.

The partnership districts have pledged to use the results of the research to adjust their own plans, Gates officials said.

“Almost all foundations think of research as evaluation that helps them look in the rear-view mirror; very few think of research as providing the headlights to help identify where are the things we could be doing that can have transformational impacts,” said Thomas Kane, the deputy director of research and data for the foundation’s education program.

The research will have several key parts, but its major thrust will be to study a variety of measures to identify effective teaching, and then to determine which combinations of those measures produce the most reliable judgments of individual teachers.

Evaluating Videos

Digital videos of teachers’ practices will be a central feature of the research. The taped observations will be scored by trained observers using a variety of performance rubrics that describe escalating levels of teacher competency. They will include the Framework for Teaching, developed by teacher-evaluation consultant Charlotte Danielson, and the Classroom Assessment Scoring System, developed by Robert C. Pianta, the dean of the education school at the University of Virginia, in Charlottesville.

Where appropriate, the videos will also be scored against several content-specific frameworks in math, English language arts, and biology.

Each teacher will be taped four times. Two of the observations will occur during lessons on common topics, and the other two will take place during randomly selected lessons. The point of that system, Mr. Kane explained, is so that researchers can determine whether similarity in lesson design helps observers’ ability to agree on a teacher’s effectiveness at presenting that lesson, or whether observers are capable of identifying high-quality teaching regardless of the context in which it occurs.

The observational data will be supplemented with student-provided information about teacher attributes that the students credit with accelerating their learning.

Overcoming Fears

Much of the foundation’s research initiative addresses teachers’ concerns about the use of standardized student test scores as a measure of teacher effectiveness.

That use of student scores has risen in prominence as an issue in part because the proposed guidelines for the $4.35 billion federal Race to the Top grant program ask states to incorporate such testing data into teacher-evaluation systems. (“Rich Prize, Restrictive Guidelines,” Aug. 12, 2009.)

Although the Gates Foundation will look at results from state reading and math tests in grades 4-8, it will supplement that information with data generated from assessments that the researchers will give to students in the various study sites and that contain open-ended, performance-based questions.

“We want to be able to take seriously the concern that people have raised that teachers that are successful at promoting achievement on the state tests aren’t necessarily the teachers that are good at promoting higher-level conceptual understanding,” Mr. Kane said.

Scholars have also raised questions about the ability of value-added assessment systems, which purport to isolate teachers’ contributions to student growth, to screen for factors that might skew estimates of teacher performance. The foundation will tackle that problem in a research component that will begin in the 2010-11 school year.

After gathering one year of effectiveness data from the various sources, researchers will randomly assign those teachers who appear to be the most effective, and those who appear to be the least effective, to specific classrooms with similar students. (The composition of the classrooms will continue to be set by school policies.)

Then, student-achievement results will be compared. The randomized design should be able to screen out statistical “noise,” organizers of the initiative say, and to verify whether the measures used accurately determine which teachers are most effective in the classroom.

Participation in the Gates research by principals and teachers will be voluntary. The teachers who participate will receive a stipend.

Douglas N. Harris, a professor of education-policy studies at the University of Wisconsin-Madison, praised the foundation’s research plans. But, he added, an equally important consideration for researchers will be to determine the appropriate use of the measures—whether for teacher improvement or for accountability purposes.

“Even if the measures are accurate, and as a result of the Gates Foundation’s work we have a better idea of who the best teachers are, it doesn’t mean that merit pay based on those measures will be effective,” he said.

A version of this article appeared in the September 16, 2009 edition of Education Week as Multi-City Study Eyes Best Gauges of Good Teaching

Events

School & District Management Webinar Examining the Evidence: Catching Kids Up at a Distance
As districts, schools, and families navigate a new normal following the abrupt end of in-person schooling this spring, students’ learning opportunities vary enormously across the nation. Access to devices and broadband internet and a secure
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
School & District Management Webinar
Branding Matters. Learn From the Pros Why and How
Branding your district matters. This webinar will provide you with practical tips and strategies to elevate your brand from three veteran professionals, each of whom has been directly responsible for building their own district’s brand.
Content provided by EdWeek Top School Jobs
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
School & District Management Webinar
How to Make Learning More Interactive From Anywhere
Nearly two-thirds of U.S. school districts are using hybrid learning right now with varying degrees of success. Students and teachers are getting restless and frustrated with online learning, making curriculum engagement difficult and disjointed. While
Content provided by Samsung

EdWeek Top School Jobs

Elementary Teacher
Madison, Wisconsin
One City Schools
Special Education Teacher
Chicago, Illinois
JCFS Chicago
Elementary Teacher - Scholars Academy
Madison, Wisconsin
One City Schools
Clinical Director
Garden Prairie, IL, US
Camelot Education

Read Next

Teaching Profession Opinion What Your Students Will Remember About You
The best teachers care about students unconditionally but, at the same time, ask them to do things they can’t yet do.
2 min read
Images shows a stylized artistic landscape with soothing colors.
Getty
Teaching Profession High Risk for COVID-19 and Forced Back to Class: One Teacher's Story
One theater teacher in Austin has a serious heart condition and cancer, but was denied the ability to work remotely. Here is her story.
9 min read
Austin High School musical theater teacher and instructional coach Annie Dragoo has three underlying health conditions noted by the CDC as being high-risk for coronavirus complications, but was denied a waiver to continue working from home in 2021.
Austin High School musical theater teacher and instructional coach Annie Dragoo has three underlying health conditions noted by the CDC as being high-risk for coronavirus complications, but was denied a waiver to continue working from home in 2021.
Julia Robinson for Education Week
Teaching Profession Photos What Education Looked Like in 2020
A visual recap of K-12 education in 2020 across the United States.
1 min read
On Sept. 24, 2020, distance learners are seen on a laptop held by teacher Kristen Giuliano who assists student Jane Wood, 11, in a seventh-grade social studies class at Dodd Middle School in Cheshire, Conn. Many schools around the state have closed temporarily during the school year because of students or staff testing positive for COVID-19. Within the first week of November 2020, nearly 700 students and more than 300 school staff around Connecticut tested positive, according to the state Department of Public Health.
Teacher Kristen Giuliano assists Jane Wood, 11, during a 7th grade social studies class in September at Dodd Middle School in Cheshire, Conn., while other students join the class remotely from home.
Dave Zajac/Record-Journal via AP
Teaching Profession Teachers Are Already Getting COVID-19 Vaccines
Some counties in Indiana began vaccinating teachers this week, ahead of schedule.
4 min read
Valerie Kelly, a 5th grade teacher in Vincennes, Ind., receives the first dose of the Pfizer COVID-19 vaccine on Dec. 28, 2020.
Valerie Kelly, a 5th grade teacher in Vincennes, Ind., receives the first dose of the Pfizer COVID-19 vaccine on Dec. 28.
Courtesy of Valerie Kelly