Special Report
Teaching Profession

E-Evaluations: ‘Watching Your Every Move’

By Michelle R. Davis — September 20, 2010 7 min read
  • Save to favorites
  • Print

Though Douglas J. Horne stands in front of a class full of students every day as a design and technology teacher at a Vermont high school, at different times he also sits before a computer to teach engineering principles to online students.

In his classroom at Essex High School in Essex Junction, Vt., an administrator might observe him a few times a year as part of his evaluation. But in Mr. Horne’s online classroom, his actions are logged and recorded and often examined by administrators at the Maynard, Mass.-based Virtual High School Global Consortium, which requires member districts, such as Essex, to supply teachers for its courses. Mr. Horne’s salary is paid by the Essex district.

“When you’re online, you almost have someone watching your every move, unlike the brick-and-mortar classroom, in which you close your door and you’re on your own,” Mr. Horne said. In virtual education, “it’s all quite visible to your students and your virtual high school administrators.”

Depth of Data

E-Learning 2010:
E-Educators Evolving

Overview: About This Report
States Eye Standards for Virtual Educators
Ed. Groups Outline E-Teacher Quality Guidelines
Teachers Make the Move to the Virtual World
Virtual Teacher Training Seen to Lack Consistency
Schools Blend Virtual and Face-to-Face Teaching
E-Educators Use Daily Mix of Digital Tools
Ed. Schools Lag Behind in Virtual Teacher Training
New Educators Look Back at Virtual-Teacher Prep.
Distinctive Demands Make Compensation Complicated
E-Evaluations: ‘Watching Your Every Move’
Web Extras
Webinar: Evaluating E-Educators’ Evolving Skills
Online Chat: Teaching in Two Worlds: Virtual And Face-to-Face
Digital Edition Read the interactive digital edition of E-Learning 2010: E-Educators Evolving.

Virtual schools are constantly collecting data on the actions their online teachers take. From the first day of each course, every chat, discussion, e-mail, grade, instant message, and keystroke a teacher makes is recorded by a virtual school’s learning-management system. Often, even a teacher’s phone calls or Skype videoconferencing sessions with students are recorded. Learning-management systems can sift through the resulting data to analyze how much time a teacher spends grading, for example, or interacting on discussion boards with students, and virtual school officials use all that information and more to evaluate instructors.

It’s far different, online experts say, from a traditional evaluation in a regular school, because of the information. The sheer depth of the data collected makes a significant difference when it comes to evaluating online teachers, said Liz R. Pape, the president of the Virtual High School Global Consortium, or VHS, a network of 660 member districts.

In a traditional evaluation scenario, an administrator “sits in on a teacher’s class two or three times a year and, of course, the teacher always polishes up those lessons so they’re shinier than bright red apples,” she said. “It’s a bit of an artificial situation. ... For the most part, once that teacher closes their classroom door, they’re on their own, doing their own thing.”

Virtual Walk-Throughs

At the Orlando-based Florida Virtual School, or FLVS - the largest state-sponsored online school in the country - virtual instructors must meet a host of expectations, including being available (though not necessarily working) to students from 8 a.m. to 8 p.m., seven days a week, said Pam Birtolo, the chief learning officer for the school, which serves about 97,000 students. The turnaround time for grading tests and assignments and replying to e-mail is 48 hours; it’s 12 hours for returning phone calls.

Students must receive specific feedback on assignments - not just a “good job.” And FLVS teachers must hold a monthly phone call with a student’s parents or guardian.

Principals examine a teacher’s electronic records monthly to view the kind of feedback, response time, and discussions instructors are having with students.

“When you’re working virtually, you do have a way to do a classroom walk-through and ensure teachers are learning about [the tone of an] e-mail, for example,” Ms. Birtolo said. “It has to be a constant give-and-take.”

Still, said Mr. Horne, the Vermont high school teacher, despite all the data collected for his online evaluation, he doesn’t feel it’s more or less accurate and objective than a face-to-face evaluation - just different. Essex High, too, collects data on teachers, around students’ graduation rates, grades, and academic achievement, for example, he said.

Online experts say it’s critical to go beyond statistics. While learning-management systems used by virtual schools can easily generate numbers on the lengths of time and ways that teachers and students interact, it’s important to look at the quality of those interactions, said Bryan H. Setser, the chief executive officer of the North Carolina Virtual Public School, which serves about 73,000 students across the state. School officials perform spot checks at least once a semester to review the transcripts of chats between a teacher and a student, for example, or listen in on a live exchange. They’ll look at how teachers are using the system’s instant-messaging tool and note that certain teachers might have 15 to 18 conversations with different students each day, while some might not use the tool for weeks.

After the spot check, a teacher will receive a green, yellow, or red rating. Green means the teacher is meeting all the criteria laid out by the school. A yellow rating triggers a conference to coach the teacher and a warning that if concerns aren’t addressed, the rating could turn to red. Teachers are put under an improvement plan when it turns to yellow and generally get one final semester “to prove they can cut the mustard,” Mr. Setser said. If that doesn’t happen, the teacher is let go.

The entire process can happen within the span of a few semesters because the school’s 345 teachers are contract workers, not bound by the same kinds of agreements that full-time public school teachers might have. In a regular public school setting, collecting the proper documentation to get rid of an unsuccessful teacher can take years, not semesters, Mr. Setser said.

“That’s what’s holding face-to-face education back,” he argued. “I’m convinced that’s part of what’s making the online environment so effective - because we don’t have those constraints.”

Student Feedback

Many online schools also turn to student feedback and completion rates as a way to evaluate teachers.

Jamey T. Fitzpatrick, the president and chief executive officer of the Michigan Virtual School, which serves online courses to about 16,000 students, said he collects student feedback during the semester, and through a detailed end-of-course survey. The survey includes queries about course rigor, teacher effectiveness, communication, and response time

The Idaho Digital Learning Academy has a lengthy teacher-evaluation process that includes a pay-for-performance bonus system. Teachers can get the bonus by meeting “exemplary expectations,” said Donna Hutchison, the chief executive officer of the state-sponsored Idaho Digital Learning Academy, or IDLA. Some of those are typical expectations for virtual educators, like fast response times, but teachers also are judged on intervention provided for struggling students and on students’ course completion.

Unlike some virtual schools, IDLA compares the course-completion rate for a teacher’s students with a three-year aggregate of that course over time.

But even supporters of online evaluations say there are some things that don’t translate in the virtual world.

In an online-only environment, for instance, it can be difficult to assess a teacher’s charisma and empathy with students, Mr. Setser said. In addition, some teachers may feel more comfortable getting constructive criticism in person.

“Some teachers find the online method somewhat impersonal,” said Mr. Setser, adding that videoconferencing can help ease that problem.

Such in-person contact can make a difference when assessing a teacher’s strengths and weaknesses and in motivating him or her to improve, said Chris Rapp, a consultant with the Evergreen, Colo.-based Evergreen Education Group and a former curriculum director at IDLA.

“It takes longer for an online principal to know their teacher personally, because they’re separated by space and time,” he said.

A version of this article appeared in the September 22, 2010 edition of Education Week as E-Evaluations: ‘Watching Your Every Move’


School Climate & Safety K-12 Essentials Forum Strengthen Students’ Connections to School
Join this free event to learn how schools are creating the space for students to form strong bonds with each other and trusted adults.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
IT Infrastructure & Management Webinar
Future-Proofing Your School's Tech Ecosystem: Strategies for Asset Tracking, Sustainability, and Budget Optimization
Gain actionable insights into effective asset management, budget optimization, and sustainable IT practices.
Content provided by Follett Learning
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Budget & Finance Webinar
Innovative Funding Models: A Deep Dive into Public-Private Partnerships
Discover how innovative funding models drive educational projects forward. Join us for insights into effective PPP implementation.
Content provided by Follett Learning

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Teaching Profession Should Class Feel Like Entertainment? Teachers Have Mixed Feelings
Teachers on social media give their opinions on whether entertaining is a necessary part of their job.
4 min read
An eighth-grade math teacher demonstrates a lesson called “math golf.”
An eighth-grade math teacher demonstrates a lesson called “math golf.”
Allison Shelley for All4Ed
Teaching Profession Teachers’ Careers Go Through Phases. They Need Support in Each
Teachers experience a dip in job satisfaction a few years into their careers.
5 min read
Vector illustration of a female teacher at her desk with her head in her hands. There are papers, stacked notebooks, and a pen on the desk and a very light photo of a blurred school hallway with bustling students walking by in the background.
Teaching Profession Download Downloadable: 5 Ways Principals Can Help With Teacher Burnout
This downloadable gives school leaders and teachers various ways to spot and treat teacher burnout.
1 min read
Silhouette of a woman with an icon of battery with low charge and icons such as a scribble line, dollar sign and lightning bolt floating around the blue background.
Teaching Profession Massages, Mammograms, and Dental Care: How One School Saves Teachers' Time
This Atlanta school offers unique onsite benefits to teachers to help them reduce stress.
3 min read
Employees learn more about health and wellness options during a mini benefits fair put on by The Lovett School in Atlanta on May 8, 2024.
Employees at the Lovett School in Atlanta meet with health benefits representatives during a mini benefits fair on May 8, 2024.
Erin Sintos for Education Week