E-Schools Put Specific Measures for Success in Place
The devil is in the details when evaluating the educational effectiveness of virtual schools
When high school students attending the full-time online WOLF school in Reno, Nev., take midterms and finals, they find themselves in an atmosphere very different from a routine school day at home.
Instead of working from the privacy of their homes—where a majority of their academic time may be spent—those students must go to a testing center, show identification, and take exams under the watchful eye of a proctor. The goal is to make sure students are doing the work and demonstrating knowledge without inappropriate assistance, says Sandi Foster, the coordinator for WOLF, which stands for Washoe School District Online Learning for the Future and is run by the 62,300-student Washoe County public school district.
"If they haven't done the rest of the work on their own, they aren't going to score well on those exams," Foster says.
Virtual schools, particularly those that provide full-time services for students, are coming under increasing scrutiny over student achievement and accountability. Several reports in recent months have questioned everything from the transient nature of virtual student populations to the integrity of student work and the lack of comparisons between online and face-to-face learning.
"I think we're flying blind on a lot of this," says Kevin G. Welner, the director of the National Education Policy Center and a professor of education policy at the University of Colorado at Boulder, where the center is based. "It's hard to know what's happening at the other end of that computer terminal."
In 2011, Welner was a co-author of the report "Online K-12 Schooling in the U.S.," which raised questions about the quality of online programs and concerns about growth and oversight of such programs. It looked mainly at schools operated by private companies.
While many of the criticisms are directed at virtual schools run by for-profit companies, even online schools affiliated with states and school districts say they are taking new steps, or reinforcing established ones, to ensure that students are learning.
At the state-financed North Carolina Virtual School, which will have about 50,000 enrollments this academic year, there's an increased concentration on accountability, says David M. Edwards, the school's chief communications and professional learning officer.
"We're focusing this year on quality and what student outcomes look like which show quality," he says.
To that end, the school is using class-size research to guide its decision to limit virtual courses to 20 to 25 students per teacher, even though other online schools may push that number closer to 100.
The school is also looking closely at mastery of concepts by students; it uses formative and summative assessments at intervals during courses to determine whether educators need to intervene. North Carolina Virtual also checks and compares performance and teacher interaction every two to three weeks, Edwards says, looking at success with assignments, assessments and test scores, and the type of exchanges between teachers and students.
In addition, North Carolina Virtual is just beginning the process of comparing its students' achievement to that of students taking the same courses in a face-to-face setting. Edwards says he believes that when the information is collected, it could have wider implications for online learning. "That data will be important on a national level as well," he says.
Checking the Data
Foster of the WOLF school also stresses the importance of data. She points to the fact that her online high school students must meet the same measures of adequate yearly progress, under the federal No Child Left Behind Act, as other Washoe district high school students in brick-and-mortar schools.
Experts say virtual schools should take some key steps to integrate accountability into their programs, including:
- Have students take exams in person to ensure they are doing their own work and absorbing information.
- Assess how students are doing numerous times throughout a course and provide intervention strategies as early as possible
- Use programs that can track detailed data on how long it takes a student to complete assignments, how often students are participating, and how frequently students and teachers have contact.
- Judge online schools by the same measures that brick-and-mortar schools are evaluated by, for example, adequate yearly progress and state testing
She points out that her school received an exemplary rating based on AYP scores, meaning students met, and a significant portion exceeded, AYP goals.
Foster says she hasn't done comparisons, however, between her students and those in face-to-face settings at nearby high schools.
That's a problem, says Welner, who believes that AYP measures provide only a rough indication of how students are doing. Those measures also don't factor in controls for the self-selective nature of online schooling, which often attracts both very high-achieving students who want to study at a faster pace and low performers who try the online alternative but still falter, he says.
"You want to be able to compare apples to apples," he says. "You're clearly getting a different group of people choosing or not choosing to go to that school."
Judy Baurenschmidt, the director of student online learning for the 84,600-student Jeffco public schools in Golden, Colo. (formerly known as the Jefferson County schools) and the director of the Jeffco 21st Century Virtual Academy, says that is an issue among her full-time virtual students. Many of the 220 or so full-time online enrollees are students who dropped out or were considered "at risk" in other schools.
Because the school serves those types of students, the district has designated her full-time virtual school as an at-risk school and compares its student-achievement data only with the performance of other schools for such students in the district.
But Bauernschmidt says her virtual school still institutes measures to help ensure students really are receiving a high-quality education compared with more traditional methods of delivery. For example, the Jefco virtual school does the same three-times-a-year benchmark testing as other schools in the district to make sure students are progressing through state and local standards.
In addition, the school assesses students as they enroll to make sure an online education is a good fit and to determine whether they need remediation in, for example, reading comprehension to be successful.
"We develop strategies and interventions at the beginning of the school year to address their needs," Baurenschmidt says. "We don't wait."
The school also runs weekly student academic-achievement reports and compares achievement between various core academic departments to see how they're stacking up against each other. Once a month, school leaders delve more deeply into data about how much interaction teachers and students have, how long it's taking students to complete assignments, and participation rates for class discussions and projects.
Beyond those measures, the school is working with its learning-management system to craft even more data tools to provide those types of measurements in real time, Baurenschmidt says.
"If it looks like a student has spent a lot of time trying to complete an assignment, that could be a red flag," she says, "so the teacher might send him an email or give him a call."
Accountability on Hold?
Despite recent discussions regarding how to measure success, some online schools have yet to move in that direction.
The state-sponsored Kentucky Virtual School, which provides supplemental online courses for students, is not "specifically accountable as a separate entity," says Lisa Gross, the director of communications for the Kentucky education department.
When students take a course, their grades and test scores are tied back to their home schools, and thus reflected in the home schools' results. The school doesn't look at completion rates or passing rates for its students, nor does it break out data in any way, Gross says.
"Our mission as a virtual program is to serve as a supplement," she says, "not as a replacement for a high school education."
Michael K. Barbour, an assistant professor of instructional technology at Wayne State University, in Detroit, says online schools should be held to the same accountability standards as brick-and-mortar schools, whether that means meeting AYP standards or state and local testing levels. Though some online schools may argue their test scores won't measure up because students start out several grade levels behind, Barbour doesn't believe that is a valid response.
"A lot of excuses they give are just excuses," he says. "These kids that are at risk or behind came from traditional brick-and-mortar schools in that position."
Vol. 31, Issue 25, Pages 12, 15Published in Print: March 15, 2012, as Measuring Success