School & District Management

Assessment to Rate Principal Leadership to Be Field-Tested

By Lynn Olson — January 11, 2008 6 min read

Starting next month, 300 schools nationwide will take part in a field test of a new way to gauge principals’ effectiveness.

Known as VAL-ED, for the Vanderbilt Assessment of Leadership in Education, the tool has been developed by a team of leadership and testing experts at Vanderbilt University and the University of Pennsylvania to measure leadership behaviors that research has found are associated with student achievement.

“Once principals are on the job, evaluation probably is the key leverage point that you need to get your hands on if you really want to move instructional leadership,” said Joseph F. Murphy, one of the developers of the new measure and a professor of education at Vanderbilt in Nashville, Tenn.

“The bottom line is superintendents can talk until their tongues fall out about learning-centered leadership, or instructional leadership,” he said, “but if they evaluate principals on criteria and standards that are different from that, principals are going to gravitate to the reality, not the language.”

VAL-ED assesses principals on six core components related to student learning, including setting high standards for achievement and creating a culture of learning and professional behavior in the school. It also measures a principal’s ability to plan, implement, support, advocate, communicate, and monitor activities in each of those areas.

The measurement tool is aligned with the standards for school leaders developed by the Interstate School Leaders Licensure Consortium, which is made up of states.

Reliable and Valid

But what will really set VAL-ED apart, according to its designers, are its psychometric properties.

While districts nationwide have developed hundreds, if not thousands, of instruments for measuring principals’ performance, few of those measures have undergone rigorous studies of their validity and reliability. That’s true despite the fact that they are often used to make high-stakes decisions, such as whether principals will receive pay raises and continue in their jobs.

“If you don’t have reliable and valid information, the inferences you’re going to make are going to be like chance,” said Andrew C. Porter, the dean of the graduate school of education at the University of Pennsylvania, located in Philadelphia. He is one of the testing experts who helped develop VAL-ED, along with Mr. Murphy; Stephen N. Elliott, who holds the Dunn Family chair in educational and psychological assessment at Vanderbilt; and Ellen Goldring, a Vanderbilt professor of education policy and leadership.

VAL-ED is being crafted with a three-year, $1.5 million grant from the New York City-based Wallace Foundation, which also supports coverage of leadership issues in Education Week.

Measuring Up

The Vanderbilt Assessment of Leadership in Education, or VAL-ED, measures how well a principal ensures six core components of schools related to student achievement are in place:

1. High standards for student learning: Individual, team, and school goals for rigorous academic and social learning are set.

2. Rigorous curriculum: Ambitious academic content is provided to all students in core academic subjects.

3. Quality instruction: Effective instructional practices maximize academic and social learning.

4. Culture of learning and professional behavior: Communities of professional practice promote student academic and social learning. A healthy school environment makes student learning the central focus.

5. Connections to external communities: Schools forge linkages to families and other people and institutions in the community that advance academic and social learning.

6. Performance accountability: Leadership holds itself and others responsible for realizing high standards of student academic and social performance. The professional staff and the school’s students exercise individual and collective responsibility

Source: Vanderbilt Assessment of Leadership in Education

“If this works, we think it will provide the first reliable way to identify principal strengths, and what their development needs are, and the ways that they’re improving over time,” said Edward Pauly, the director of research and evaluation for the foundation, “all of which would be extremely valuable to people in the field.”

In preparation for designing VAL-ED, the team gathered about 65 principal evaluations now being used in big-city districts. Only two of those, said Mr. Porter, provided any documentation of their psychometric properties.

In contrast, VAL-ED has been subject to a number of traditional studies used to develop educational measurements. In one study, principals were asked to sort each item according to the leadership component it was intended to measure (they did so with 80 percent accuracy). Researchers also conducted two rounds of cognitive interviews, in which individuals thought out loud as they completed the evaluation. And they conducted a nine-school pilot test last spring, which found the instrument had high rates of reliability (0.95 for the total score, and about 0.90 for each subscale).

Field Tests

Dennis S. Perry, the principal of the 950-student Henry C. Beck Middle School, in Cherry Hill, N.J., is one of those who took the survey instrument and provided the researchers with feedback.

“I think that it’s going to be an effective instrument for quickly gathering data for the evaluation of administrators,” he said. “Any tool that will help to provide some sort of consistency, as far as what the skill sets necessary for administrators are, I think will be an effective tool.”

The 300-school field test, to be completed during February and March, will replicate some of the earlier studies. It will include 100 elementary, 100 middle, and 100 high schools across 53 districts in 27 states. The field test also will be used to allow VAL-ED to report results in two ways: how a principal performs compared with his or her peers nationwide, and whether he or she performs at a “basic,” “proficient,” or “distinguished” level, based on expert judgment about what good-enough performance looks like.

The actual instrument consists of 72 items, which participants can complete in about 20 to 25 minutes. The tool is designed to give principals “360-degree” feedback about their performance: All the teachers at a school, the principal, and his or her supervisor complete the evaluation.

Each individual rates the principal’s performance on 72 behaviors, from 1 for “ineffective,” to 5 for “outstandingly effective,” after first being asked to consider the sources of evidence on which the rating is based, such as personal observations or school documents. A typical item might ask, for example, “How effective is the principal at ensuring the school evaluates the rigor of the curriculum?”

Districts will be able to choose whether to use VAL-ED annually or more often, online or in paper-and- pencil formats. Two parallel forms facilitate using VAL-ED to measure a principal’s growth over time. Districts also can decide how much weight to give the judgments of teachers compared with that of the principal’s supervisor; how to use VAL-ED in combination with other information; and whether to measure how a principal performs compared with his or her peers nationally or against a standard.

Making Judgments

While the results can be used to provide principals with ongoing feedback, and to guide their professional development, “it’s really a tool to help make judgments about the quality of personnel,” Vanderbilt’s Mr. Murphy said. “That’s why it’s an evaluation.”

“You can imagine districts using it for principal renewal, for the length of contracts, for incentive pay,” he said.

The estimated cost is about $350 per school.

But the researchers are quick to point out that while VAL-ED may be an improvement over previous evaluation instruments, it also has its limits.

Essentially, it asks for people’s perceptions of a principal’s leadership behaviors. It does not measure student-learning gains in the school, customer satisfaction, or the actual presence of the core components in the school building, such as a rigorous curriculum or high-quality instruction.

Eventually, Mr. Murphy said, a robust evaluation system would include all those measures.

“To assess the strengths and weaknesses of a principal, you really should look beyond just what you learn from our instrument,” agreed Mr. Porter of the University of Pennsylvania. “It would be ideal if you had direct measures of the core components— if you had, for example, a direct measure of the rigor of the curriculum. Also, if you had a direct measure of the value added to student achievement.

“So we’ll be promoting that our instrument be used in conjunction with these other pieces of information.”

A version of this article appeared in the January 16, 2008 edition of Education Week as Assessment to Rate Principal Leadership To Be Field-Tested


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Teaching Webinar
Interactive Learning Best Practices: Creative Ways Interactive Displays Engage Students
Students and teachers alike struggle in our newly hybrid world where learning takes place partly on-site and partly online. Focus, engagement, and motivation have become big concerns in this transition. In this webinar, we will
Content provided by Samsung
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Classroom Technology Webinar
Educator-Driven EdTech Design: Help Shape the Future of Classroom Technology
Join us for a collaborative workshop where you will get a live demo of GoGuardian Teacher, including seamless new integrations with Google Classroom, and participate in an interactive design exercise building a feature based on
Content provided by GoGuardian
School & District Management Live Online Discussion A Seat at the Table With Education Week: What Did We Learn About Schooling Models This Year?
After a year of living with the pandemic, what schooling models might we turn to as we look ahead to improve the student learning experience? Could year-round schooling be one of them? What about online

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management From Our Research Center How the Pandemic Is Shaping K-12 Education (in Charts)
Surveys by the EdWeek Research Center show how schools have changed during the pandemic and what adjustments are likely to stick.
Eric DiVito gives breathing instructions as he teaches a remote music class at the Osborn School on Oct. 6, 2020, in Rye, N.Y.
Eric DiVito gives breathing instructions as he teaches a remote music class at the Osborn School in Rye, N.Y., last fall.
Mary Altaffer/AP
School & District Management 'You Can’t Follow CDC Guidelines': What Schools Really Look Like During COVID-19
All year, some teachers have said that enforcing precautions to slow the spread of the virus in classrooms can be nearly impossible.
13 min read
Guntown Middle School eighth graders walk the halls to their next class as others wait in their assigned spots against the wall before moving into their next class during the first day back to school for the Lee County District in Guntown, Miss on Aug. 6, 2020.
Eight graders walk the halls on the first day back to school in Guntown, Miss., on Aug. 6, 2020. Teachers in several states told Education Week that since the beginning of the school year, enforcing precautions such as social distancing to slow the spread of the coronavirus has been nearly impossible.<br/>
Adam Robison/The Northeast Mississippi Daily Journal via AP
School & District Management Opinion School Reopening Requires More Than Just Following the Science
Educators can only “follow the science” so far. Professional expertise matters too, writes Susan Moore Johnson.
Susan Moore Johnson
5 min read
Illustration of school and bus
School & District Management Why Teacher Vaccinations Are So Hard to Track
Teachers can now get the COVID-19 vaccine, but there’s no way of knowing how many are currently inoculated against the virus.
6 min read
Image of a needle and vaccine bottle.