School & District Management

Data Fall Short When Evaluating Impacts of Principal-Prep Programs, Report Finds

By Denisa R. Superville — November 03, 2016 5 min read
  • Save to favorites
  • Print

In a two-year effort to examine how the graduates of five principal-preparation programs affected student outcomes once they were on the job, researchers came up against a major challenge: Good data were hard to come by.

But despite the dearth of good data, the researchers with the George W. Bush Institute and the American Institutes for Research stressed that understanding whether principal preparation programs are training principals for the work conditions and challenges of the job is an “urgent” policy concern. That’s particularly true, the study’s authors said, because of how much principals’ jobs have changed in recent years, making it more important than ever to track principals during their careers and analyze the outcomes of their leadership on their schools and the students who attend them.

In their quest to do that, the data were a persistent problem. Sample size was an issue. Principal preparation programs graduate small numbers of students and not all go into the principalship right away. The researchers sought three years of student achievement data for their report, but because turnover in the profession is high—nearly half of new principals do not stay beyond three years, according to a study from the School Leaders Network—some schools and principals had to be excluded. Another limitation researchers encountered: districts that generally don’t keep track of their principals’ experiences before they arrive at the district or where those principals were trained.

Still, despite those challenges, the researchers found some illuminating things to note, including:

  • While principal graduates provided generally positive reviews of the course work at their preparation programs and the hands-on experiences they received, they had a dimmer view of the support they received from the districts and their programs once they were on the job.

  • The prep programs in the study produced graduates who were both high-performing and low-performing.

The findings, they noted, suggest that “focusing on reducing variation in the performance of graduates through training, selection, or other means and systematizing or better tailoring supports may be the keys to success in preparing effective school leaders.”

The report, “Developing Leaders: The Importance—And The Challenges—Of Evaluating Principal Preparation Programs,” comes as principal-prep programs are under more pressure to improve, superintendents say they are dissatisfied with their principals’ readiness, and new, more rigorous education standards are being rolled out.

Title II ESSA guidelines, published by the U.S. Department of Education in September, provide new opportunities for states and districts to improve (and rethink) principal preparation and on-the-job support for school leaders.

Earlier this year, a report from the Wallace Foundation on improving university-based principal-preparation programs included this important nugget: 80 percent of school district superintendents thought that principal-prep programs need to be improved. Wallace recently announced a $47 million grant to districts and universities in seven states to revamp principal-prep programs and work with those states to take a more muscular role in improving program quality by, among other things, reviewing and revamping their licensing and certification requirements. (The George W. Bush Institute/AIR study looked at both university-based and non-profit prep programs.)

This latest prep-program evaluation is part of a series of school leadership research the George W. Bush Institute released last week in conjunction with AIR.

Another of its reports explored the importance of talent management, and provided examples of how school systems in Denver; Gwinnett County, Ga.; Houston; and Charlotte-Mecklenburg County, N.C., were using what it called a “framework” for talent management, a system that starts with preparation programs and continues with professional learning and incentives for principals on the job, in an effort to strengthen their principal pipelines.

Representatives from the Hillsborough County, Fla., school system and Charlotte-Mecklenburg district will be among the participants in a panel discussion at the American Institutes for Research in Washington on Nov. 3 on the “framework” and how their districts are using it.

Preparation programs—including the requirements and standards they must meet to stay open—vary from state to state, and few programs are evaluated based on their graduates’ impact on improving student achievement, according to the new paper.

On the principal prep-study, the researchers’ report suggests considering other factors such as teacher retention and school climate in addition to student achievement when seeking to measure a principal’s impact. But the report also noted that even those measures come with their own set of problems. Teacher-retention, for instance, is an incomplete measure because one way that a principal can improve his or her school is to get rid of ineffective teachers.

It is also difficult to “disentangle” how much of a principal’s impact is due to his or her preparation program or skills he or she picked up along the way, from their principal supervisors or through district support and professional development, the report noted. And, the researchers noted, it takes time for principals to have an impact on their schools.

Again, the researchers returned to shortcomings in the data.

The report makes recommendations, stressing that prep programs need to know how their graduates were affecting student achievement in order to make continuous improvements and that districts need to know which programs were producing the best principals.

Among the recommendations: States and districts should collect more systematic data on student outcomes in addition to standardized tests; embrace programs that use best practices recommended by experts; evaluate and collect more data on the programs that are producing the best graduates; conduct additional research on developing and retaining the most effective principals; and continue to draw lessons from leadership research in other fields.

You can dig into all of the George W. Bush Institute reports here:

A version of this news article first appeared in the District Dossier blog.

Commenting has been disabled on effective Sept. 8. Please visit our FAQ section for more details. To get in touch with us visit our contact page, follow us on social media, or submit a Letter to the Editor.


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Teaching Webinar
6 Key Trends in Teaching and Learning
As we enter the third school year affected by the pandemic—and a return to the classroom for many—we come better prepared, but questions remain. How will the last year impact teaching and learning this school
Content provided by Instructure
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
School & District Management Webinar
Ensuring Continuity of Learning: How to Prepare for the Next Disruption
Across the country, K-12 schools and districts are, again, considering how to ensure effective continuity of learning in the face of emerging COVID variants, politicized debates, and more. Learn from Alexandria City Public Schools superintendent
Content provided by Class
Teaching Profession Live Online Discussion What Have We Learned From Teachers During the Pandemic?
University of California, Santa Cruz, researcher Lora Bartlett and her colleagues spent months studying how the pandemic affected classroom teachers. We will discuss the takeaways from her research not only for teachers, but also for

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management Some Teachers Won't Get Vaccinated, Even With a Mandate. What Should Schools Do About It?
Vaccine requirements for teachers are gaining traction, but the logistics of upholding them are complicated.
9 min read
Illustration of a vaccine, medical equipment, a clock and a calendar with a date marked in red.
School & District Management A Vaccine for Kids Is Coming. 6 Tips for Administering the Shot in Your School
Start planning now, get help, and build enthusiasm. It's harder than it looks.
11 min read
Cole Rodriguez, a 15-year-old student at Topeka West, gets a COVID-19 vaccine Monday, Aug. 9, 2021 at Topeka High School's vaccine clinic.
Cole Rodriguez, a 15-year-old student, gets a COVID-19 vaccine at Topeka High School's vaccine clinic.
Evert Nelson/The Topeka Capital-Journal via AP
School & District Management Letter to the Editor School Mask Mandates: Pandemic, ‘Panicdemic,’ or Personal?
"A pandemic is based on facts. A 'panicdemic' is based on fears. Today, we have both," writes a professor.
1 min read
School & District Management How 'Vaccine Discrimination' Laws Make It Harder for Schools to Limit COVID Spread
In Montana and Ohio, the unvaccinated are a protected class, making it tough to track and contain outbreaks, school leaders say.
4 min read
Principal and District Superintendent Bonnie Lower takes the temperature of a student at Willow Creek School as the school reopened, Thursday, May 7, 2020, in Willow Creek, Mont.
Bonnie Lower, a principal and district superintendent in Willow Creek, Mont., checks the temperature of a student as Willow Creek School reopened for in-person instruction in the spring.
Ryan Berry/Bozeman Daily Chronicle via AP