Teaching Profession

Student Achievement, Observations Correlated in Chicago Pilot

By Stephen Sawchuk — November 16, 2011 3 min read
  • Save to favorites
  • Print

Both a value-added method and principal observations tied to a teaching framework identified the same teachers as particularly high or low-performing under Chicago’s teacher-evaluation pilot, a new study concludes.

But principals struggled to provide high-quality “coaching” and support to teachers based on the results, the report says—a finding that gives a sense of just how difficult it will be to use the new teacher evaluation systems springing up across the nation to improve teaching and learning.

Released last weekend, the report from the Consortium for Chicago School Research is the year two analysis of Chicago’s pilot teacher evaluation system (follow the embedded link here to read my write-up of the year one findings.)

In brief, the study looks at the performance of teachers in select elementary schools in which the system was piloted over the period of 2008-09 to 2009-10. The system uses the Framework for Teaching developed by Charlotte Danielson.

To determine reliability, the study’s authors compared ratings given by principals with those of external evaluators. To look at the system’s validity, they analyzed the relationship between principals’ observations and value-added estimates for the teachers, where available.

Value-added, you’ll remember, is a statistical technique used to estimate how much growth an average student makes in a year and then compares that to actual performance, the idea being that those students who made more or less than the predicted amount must have had a particularly good or weak teacher.

Finally, the authors conducted case studies in eight of the pilot elementary schools.

Here’s a rundown of the findings, plus a discussion to help you situate them within the literature on teacher evaluations.

• Across most of the framework standards, students showed the greatest growth in test scores in the classrooms in which teachers received the highest observation ratings from their principals, and the least growth in those where they received the lowest ratings. This finding is similar to the conclusion of several studies on Cincinnati’s teacher-evaluation system.

• Principals and observers gave similar numbers of lower scores, but principals gave the top rating more often than the other observers did, across all 10 of the evaluations standards. Interestingly, much of this variation disappeared when researchers controlled for the teachers’ prior evaluation scores, suggesting that principals may be drawing on background knowledge in assigning scores. While this doesn’t exactly fit the narrative of vindictive principals, it does show that who you get as an observer potentially matters.

• Most of the principals were close to the external observers in terms of how strictly they applied the evaluation standards, but there were a few outliers on both ends. Eleven percent of principals regularly rated teachers lower than the observers while 17 percent tended to rate them higher. Another reason to consider more than one observer in a teacher-evaluation system.

• The case studies revealed that teachers and principals generally reported more meaningful discussions about teaching practice than the old “checklist” evaluation did, and that the conversations were more evidence-based. Teachers’ experiences were largely dependent on whether they felt the principal had applied the framework fairly.

• Principals struggled to ask high-level, complex questions during evaluation conferences to elicit better reflection from teachers on their scores. Principals felt they weren’t given enough training on this “coaching” aspect, and they tended to dominate the conversations.

Much more in the report, so check it out. In the meantime, it will be interesting to see how this pilot informs the development of a new teacher evaluation system in the city, which has lurched back and forth now between a couple of different frameworks. We’ll be watching here with interest.

A version of this news article first appeared in the Teacher Beat blog.


Commenting has been disabled on edweek.org effective Sept. 8. Please visit our FAQ section for more details. To get in touch with us visit our contact page, follow us on social media, or submit a Letter to the Editor.


Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Teaching Webinar
6 Key Trends in Teaching and Learning
As we enter the third school year affected by the pandemic—and a return to the classroom for many—we come better prepared, but questions remain. How will the last year impact teaching and learning this school
Content provided by Instructure
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
Attendance Awareness Month: The Research Behind Effective Interventions
More than a year has passed since American schools were abruptly closed to halt the spread of COVID-19. Many children have been out of regular school for most, or even all, of that time. Some
Content provided by AllHere
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
School & District Management Webinar
Ensuring Continuity of Learning: How to Prepare for the Next Disruption
Across the country, K-12 schools and districts are, again, considering how to ensure effective continuity of learning in the face of emerging COVID variants, politicized debates, and more. Learn from Alexandria City Public Schools superintendent
Content provided by Class

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Teaching Profession Opinion ‘A Culture of Care’: How Schools Can Alleviate Educator Stress This Year
It takes more than deep breathing to alleviate the stress teachers feel. Here's how to get to the root cause.
Sean Slade & Alyssa Gallagher
6 min read
shutterstock 740616958 resized
Shutterstock
Teaching Profession Reported Essay Students Aren’t the Only Ones Grieving
Faced with so many losses stemming from the pandemic, what can be done to help teachers manage their own grief?
4 min read
Conceptual Illustration
Pep Montserrat for Education Week
Teaching Profession We Feel Your Grief: Remembering the 1,000 Plus Educators Who've Died of COVID-19
The heartbreaking tally of lives lost to the coronavirus continues to rise and take a steep toll on school communities.
3 min read
090321 1000 Educators Lost BS
Education Week
Teaching Profession Letter to the Editor Educators Have a Responsibility to Support the Common Good
A science teacher responds to another science teacher's hesitation to take the COVID-19 vaccine.
1 min read