Education

Evaluating Principals Through Test Scores: Harder Than You’d Think

By Jaclyn Zubrzycki — December 17, 2012 3 min read
  • Save to favorites
  • Print

New research attempts to determine the best way to measure principal effectiveness using students’ test scores—and finds that the task is trickier than anticipated.

Principal evaluation presents a number of questions: Should principals be evaluated based on the performance of teachers they didn’t hire? Should they be measured for their immediate impact or for growth over time? How can we actually compare one principal to another working in an entirely different school or district context?

In the new research, Jason A. Grissom at Vanderbilt University and Demetra Kolgrides and Susanna Loeb at Stanford University propose and examine several possible methods of using test scores to evaluate principals, adjusting, in each case, for the background characteristics of the students that might affect academic performance. They analyze three broad approaches to evaluation: tying principal performance directly to schools’ performance (“school effectiveness”); comparing different principals’ performance at the same school (“relative within-school effectiveness”); and examining growth in student achievement over a principal’s tenure (“school improvement”).

Tying principal performance directly to test scores (“school effectiveness”) may seem too blunt. “You’re capturing principal performance, but you’re capturing a number of other things, too,” Grissom said in an interview. And yet, in the end, the researchers found that this measure aligned most closely with what the researchers called “nontest measures of performance,” like surveys of parents. However, this may mean that those outside indicators were reflecting the general performance of the school more than the performance of the principal, Grissom said.

The other methods may be theoretically more appealing, as they attempt to narrow in on what the principal actually controls. But they present logistical and practical challenges—and the results, the researchers write,"inspire less confidence.” Comparing different principals’ performance at the same schools is difficult because principals don’t leave their jobs frequently enough to compare accurately between principals. (Nor would we want them to!) Assessing principals’ growth over time might be more effective, but it requires a large amount of data and, in this run-through, was not particularly reliable. Neither of these measures matched up well with the “nontest measures of performance.” They also didn’t match up with each other.

So, in the end, this research doesn’t necessarily clarify matters for policymakers looking to come up with effective principal evaluation tools. The paper concludes with a caution: “The warning that comes from these analyses is that it is important to think carefully about what the measures are revealing about the specific contribution of the principal and to use the measures for what they are, which is not as a clear indicator of principals’ specific contributions to student test score growth.”

But policymakers are moving toward tying principal evaluation and effectiveness to student achievement. Several states, including Florida, where this research took place, have laws requiring principal evaluations to be tied to students’ test scores.

Grissom said in an interview that this doesn’t mean that districts should avoid looking at test scores at all. “It means that you have to be thoughtful about it, and cognizant of what you’re measuring. And you want to use it in conjunction with other sources of data if you’re going to use it for accountability,” he said. That lines up with recommendations the two national principal associations put out earlier this year.

A look at the effectiveness of evaluation systems actually in place in large districts or in states that mandate this kind of connection could be revealing. This study’s data comes from the Miami-Dade public school system 2003-2011. Grissom says that he and his colleagues will likely look at systems in practice in Miami and potentially elsewhere.

Meanwhile, principal effectiveness will be the topic of an upcoming forum and public discussion hosted by the ASCD, as we shared last week.

Want to keep up with school district and leadership news? Follow @district_doss on Twitter.

A version of this news article first appeared in the District Dossier blog.


Commenting has been disabled on edweek.org effective Sept. 8. Please visit our FAQ section for more details. To get in touch with us visit our contact page, follow us on social media, or submit a Letter to the Editor.


Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Teaching Webinar
6 Key Trends in Teaching and Learning
As we enter the third school year affected by the pandemic—and a return to the classroom for many—we come better prepared, but questions remain. How will the last year impact teaching and learning this school
Content provided by Instructure
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Curriculum Webinar
How Data and Digital Curriculum Can Drive Personalized Instruction
As we return from an abnormal year, it’s an educator’s top priority to make sure the lessons learned under adversity positively impact students during the new school year. Digital curriculum has emerged from the pandemic
Content provided by Kiddom
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Equity & Diversity Webinar
Leadership for Racial Equity in Schools and Beyond
While the COVID-19 pandemic continues to reveal systemic racial disparities in educational opportunity, there are revelations to which we can and must respond. Through conscientious efforts, using an intentional focus on race, school leaders can
Content provided by Corwin

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Education More States Are Requiring Schools to Teach Native American History and Culture
Advocates say their efforts have gained some momentum with the nation’s reckoning over racial injustice since the killing of George Floyd.
3 min read
A dancer participates in an intertribal dance at Schemitzun on the Mashantucket Pequot Reservation in Mashantucket, Conn., Saturday, Aug. 28, 2021. Connecticut and a handful of other states have recently decided to mandate students be taught about Native American culture and history. (AP Photo/Jessica Hill)
Education Judge's Temporary Order Allows Iowa Schools to Mandate Masks
A federal judge ordered the state to immediately halt enforcement of a law that prevents school boards from ordering masks to be worn.
4 min read
Iowa Gov. Kim Reynolds speaks to reporters following a news conference, Thursday, Aug. 19, 2021, in West Des Moines, Iowa. Reynolds lashed out at President Joe Biden Thursday after he ordered his education secretary to explore possible legal action against states that have blocked school mask mandates and other public health measures meant to protect students against COVID-19. Reynolds, a Republican, has signed a bill into law that prohibits school officials from requiring masks, raising concerns as delta variant virus cases climb across the state and schools resume classes soon. (AP Photo/Charlie Neibergall)
Education Hurricane Ida Deals New Blow to Louisiana Schools Struggling to Reopen
The opening of the school year offered teachers a chance to fully assess the pandemic's effects, only to have students forced out again.
8 min read
Six-year-old Mary-Louise Lacobon sits on a fallen tree beside the remnants of her family's home destroyed by Hurricane Ida, in Dulac, La., on Sept. 4, 2021. Louisiana students, who were back in class after a year and a half of COVID-19 disruptions kept many of them at home, are now missing school again after Hurricane Ida. A quarter-million public school students statewide have no school to report to, though top educators are promising a return is, at most, weeks away, not months.
Six-year-old Mary-Louise Lacobon sits on a fallen tree beside the remnants of her family's home destroyed by Hurricane Ida, in Dulac, La., on Sept. 4, 2021.
John Locher/AP
Education Massachusetts National Guard to Help With Busing Students to School
250 guard personnel will be available to serve as drivers of school transport vans, as districts nationwide struggle to hire enough drivers.
1 min read
Massachusetts National Guard soldiers help with logistics in this Friday, April 17, 2020 file photo, at a food distribution site outside City Hall, in Chelsea, Mass. Mass. Gov. Charlie Baker on Monday, Sept. 13, 2021, activated the state's National Guard to help with busing students to school as districts across the country struggle to hire enough drivers.
Massachusetts National Guard soldiers help with logistics in this Friday, April 17, 2020 file photo, at a food distribution site outside City Hall, in Chelsea, Mass.
Michael Dwyer/AP