School & District Management

Review Backs New Tool for Principal Evaluation

By Lesli A. Maxwell — December 22, 2009 4 min read

An examination of how to size up the performance of principals has found that one evaluation method is best suited for judging the effectiveness of school leaders: the Vanderbilt Assessment of Leadership in Education.

Created in 2006 and just now widely available for districts to purchase, the assessment, called VAL-ED, is the newest of the principal instruments in the review conducted by Matthew Clifford and Christopher Condon, researchers with Learning Point Associates, a nonprofit educational consulting firm based in suburban Chicago. (“Assessment to Rate Principal Leadership to Be Field-Tested,” Jan. 16, 2008.)

Mr. Clifford said the impetus for the review was the growing recognition that principals, second only to classroom teachers, affect student learning. Districts need as much information as possible to make high-stakes decisions about whom they hire as principals, how or whether they should invest in their improvement, and how to compensate them, he noted.

“Districts face critical choices at three points in the principal’s life with them: at hiring, at the point of evaluation, and at the point of deciding in some cases whether principals receive tenure or advanced certification,” Mr. Clifford said. “They need the best information they can get for these purposes.”

In reviewing eight principal-performance instruments being used by school districts, the researchers concluded that VAL-ED comes closest to measuring the leadership attributes and behaviors that research finds to be associated with how well students perform. VAL-ED also was rated the best among the instruments for validity and reliability, meaning that the assessment measures what it is supposed to measure and yields consistent results.

Measuring Leadership

The analysis by Learning Point Associates looked at the following evaluations.

INSTRUMENT:
Change Facilitator Style Questionnaire (1988)

APPROACH:
• 77-item assessment that addresses six domains
• Collects information about teachers’ view of the principal as leader

INSTRUMENT:
Diagnostic Assessment of School and Principal Effectiveness (1992)

APPROACH:
• 360-degree evaluation focusing on educational leadership
• 213-item survey independently completed by students, teachers, principals, and others
• Results combined into a score

INSTRUMENT:
Instructional Activity Questionnaire (1987)

APPROACH:
• A 34-item assessment
• Covers three subscales that specifically address instructional leadership

INSTRUMENT:
Leadership Practices Inventory (2002)

APPROACH:
• 30-item measure of general leadership practices to be completed by the principal and an observer or supervisor
• Widely used to measure leadership effectiveness

INSTRUMENT:
Performance Review Analysis and Improvement System for Education (1985)

APPROACH:
• 81-item assessment that includes nine subscales
• Produces a two-dimensional leadership profile and identification of strengths and weaknesses

INSTRUMENT:
Principal Instructional Management-Rating Scale (1985)

APPROACH:
• 71-item questionnaire that addresses 11 educational-leadership subscales
• Widely used in the field

INSTRUMENT:
Principal Profile (1986 and 1987)

APPROACH:
• Interview-based assessment technique that measures leadership effectiveness on certain tasks and characterizes leadership style
• Used primarily as a diagnostic tool

INSTRUMENT:
Vanderbilt Assessment of Leadership in Education (2006)

APPROACH:
• 360-degree assessment tool to be administered to principals, teachers, and principals’ supervisors
• Consists of 72 items
• Produces a quantitative diagnostic profile
• Linked to national standards

SOURCE: Learning Point Associates

Only one other instrument the researchers reviewed was developed in this decade. The remaining six date back as far as 1985.

The researchers liked the “360-degree” approach of VAL-ED, which gives principals feedback about their performance from all the teachers in their school and from the principals’ supervisors. Only one other instrument in the review used a similar approach for providing multiple sources of input into an evaluation.

“What I think our findings suggest is that we need to invest more in ensuring that these assessments measure what is considered important for school leaders now,” said Mr. Clifford, who is a senior research and policy associate at Learning Point. “The demands of the job have changed a lot.”

New Focus to Job

Joseph F. Murphy, a developer of VAL-ED and an education professor at Vanderbilt University in Nashville, Tenn., said the instrument was designed because no existing evaluation system was effectively measuring what had become the most important role for principals: instructional leadership.

“The profession for 150 years was grounded in management, organization, government, politics, and finance,” Mr. Murphy said. “Those things are important, but they are secondary to learning and teaching.”

Even though the expectations for the job have shifted and broad agreement exists in the field on the principal’s most important role, most school leaders are still being evaluated on two things, Mr. Murphy said: controlling conflict and running smooth operations.

“That’s the reality,” he said. “Until principals know they are going to be evaluated on things like poking their nose into a teacher’s business, most of them aren’t going to do it. What we’ve argued is that once a principal is on the job, the strongest leverage point you have to shape their behavior is in the evaluation.”

VAL-ED is now a product sold by the Nashville-based company Discovery Education, a division of Discovery Communications. It became available for widespread commercial use for the current school year, Mr. Murphy said. He does not know how many districts have implemented the system so far, but said school districts in states such as Alabama, Louisiana, and Mississippi are adopting it on a fairly broad scale.

“Most districts are still using some sort of homegrown evaluation tool for principals,” he said.

Indeed, there are probably hundreds, if not thousands, of evaluation approaches at work in schools, said Mr. Clifford. To decide which ones should be included, he and Mr. Condon limited their review to those instruments that were expressly intended for assessment purposes, were publicly available for purchase, and had been psychometrically tested for validity and reliability. The researchers did not collect data to determine how widespread the use of each instrument is.

The researchers found promising features in each of the eight approaches, Mr. Clifford said.

For example, the Performance Review Analysis and Improvement System for Education, or PRAISE, which dates to 1985, helps each leader “get a profile of himself or herself and helps chart the next steps in professional development,” said Mr. Clifford. It’s also practical, in that it takes only 15 to 20 minutes to answer the 81-item assessment.

“The content could be dated at this point, but we think it has potential,” he said.

Based on their findings, however, Mr. Clifford said VAL-ED is probably the best evaluation tool available. Still, he cautioned that the quality of an assessment on paper has little bearing on how effective it is in practice.

“If it’s a poorly administered assessment, then the data coming back won’t be good,” he said. “The practice of evaluation is as critical as the design.”

A version of this article appeared in the January 06, 2010 edition of Education Week as Review Finds Principal-Evaluation Tools a Bit Outdated

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Professional Development Webinar
Building Leadership Excellence Through Instructional Coaching
Join this webinar for a discussion on instructional coaching and ways you can link your implement or build on your program.
Content provided by Whetstone Education/SchoolMint
Teaching Webinar Tips for Better Hybrid Learning: Ask the Experts What Works
Register and ask your questions about hybrid learning to our expert panel.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Families & the Community Webinar
Family Engagement for Student Success With Dr. Karen Mapp
Register for this free webinar to learn how to empower and engage families for student success featuring Karen L. Mapp.
Content provided by Panorama Education & PowerMyLearning

EdWeek Top School Jobs

Hiring Bilingual and Special Education Teachers NOW!
Newark, New Jersey
Newark Public Schools
DevOps Engineer
Portland, OR, US
Northwest Evaluation Association
User Experience Analyst
Portland, OR, US
Northwest Evaluation Association
Senior Business Analyst - 12 Month Contract
Portland, OR, US
Northwest Evaluation Association

Read Next

School & District Management One District's COVID-19 Journey: A Year of Upheaval and Unexpected Insights
The Everett, Wash., school district was among the first in the nation shuttered by the pandemic. Educators and parents share and reflect.
9 min read
School & District Management The Key to School-Based COVID-19 Testing: Cooperation of Parents and Communities
As schools launch broad testing to track cases of COVID-19, the success of their efforts relies on addressing the concerns of all concerned.
7 min read
Katie Ramirez, left, watches as her mother, Claudia Campos, swabs the mouth of her sister, Hailey, for a COVID-19 test at a testing site in Los Angeles on Dec. 9, 2020.
Katie Ramirez, left, watches as her mother, Claudia Campos, swabs the mouth of her sister, Hailey, for a COVID-19 test at a testing site in Los Angeles.
Jae C. Hong/AP
School & District Management Interactive A Year of COVID-19: What It Looked Like for Schools
This timeline offers a look at how a full year of living and learning during the COVID-19 pandemic unfolded.
Education Week Staff
13 min read
Elementary 1 teacher Melissa Vozar sits outside of Suder Elementary in Chicago to teach a virtual class on Jan. 11, 2021. The Chicago Teachers Union said that its members voted to defy an order to return to the classroom before they are vaccinated against the coronavirus, setting up a showdown with district officials who have said such a move would amount to an illegal strike.
Elementary 1 teacher Melissa Vozar sits outside of Suder Elementary in Chicago to teach a virtual class on Jan. 11, 2021. The Chicago Teachers Union said that its members voted to defy an order to return to the classroom before they are vaccinated against the coronavirus, setting up a showdown with district officials who have said such a move would amount to an illegal strike.
Anthony Vazquez/Chicago Sun-Times via AP
School & District Management Most Principals, District Leaders Predict Their Schools Will Be Fully In-Person This Fall
EdWeek Research Center surveys track the growing trend to get more students back in school buildings as soon as possible.
5 min read
Assistant Principal Janette Van Gelderen, left, welcomes students at Newhall Elementary in Santa Clarita, Calif on Feb. 25, 2021. California's public schools could get $6.6 billion from the state Legislature if they return to in-person instruction by the end of March, according to a new agreement announced Monday, March 1, 2021, between Gov. Gavin Newsom and the state's legislative leaders.
Assistant Principal Janette Van Gelderen, left, welcomes students at Newhall Elementary in Santa Clarita, Calif., last month. California's public schools could get $6.6 billion from the state if they return to in-person instruction by the end of March.
Marcio Jose Sanchez/AP