Special Report
Classroom Technology

Quality Control a Challenge for Virtual Ed.

By Ian Quillen — August 27, 2012 6 min read
  • Save to favorites
  • Print

As online learning has entered the mainstream—with roughly a third of the nation’s high school students enrolled in at least one online course, according to a report released in June 2011—more states have created policies, procedures, and even organizations for evaluating the quality of such courses and other online content available to students.

But instituting those quality-control measures is not without challenges.

For starters, the creation of such systems often requires legislation, which must be crafted by lawmakers who aren’t always familiar with online learning’s nuances and may not be eager to launch new programs in tight budget times.

Further, the peculiarities of individual states’ education policies—especially on standards and materials approval—mean the kind of oversight that might work effectively in one state might not function well in another.

“There’s no state we could look at and say they’ve got it all figured out,” said John Watson, the founder of the Evergreen Education Group, the Durango, Colo.-based research firm that puts out the annual “Keeping Pace” reports measuring growth and changes in K-12 online learning. Taking a functional system in one state and applying it to the policy realities of another would be “fairly difficult,” Mr. Watson added.

Even the California Learning Resource Network, or CLRN, and the Texas Virtual School Network, or TXVSN, which partnered to write a second edition of the iNACOL National Standards for Quality Online Courses, released last October, have considerably different authority, responsibilities, and procedures. (The Vienna, Va.-based International Association for K-12 Online Learning, or iNACOL, is a leading advocacy group for proponents of online and blended learning for students.)

The Houston-based TXVSN, for example, must approve all courses offered to regular public or charter school students in Texas, whether the classes are a supplement to brick-and-mortar coursework or part of a full-time virtual program.

By contrast, the Modesto-based CLRN acts in a more advisory role to districts and charter schools seeking to ensure that they purchase high-quality content, said CLRN Director Brian Bridges. The organization can’t deny districts or charters the right to use their online courses of choice. However, CLRN was set this month to announce a new partnership with the University of California system in which only CLRN-approved online courses will be accepted as valid when state universities evaluate high school students’ transcripts.

Further, school districts and state universities are the main creators of the content reviewed by the TXVSN, while CLRN can review content created by districts, universities, or private companies. And unlike CLRN, the Texas group takes ownership of the courses it reviews and acts as the vendor for interested districts or charter schools.

Even the organizations’ roots are different; legislators in 2007 created the TXVSN explicitly for reviewing and approving online-course content. By contrast, the California group has morphed into an organization focused primarily on reviewing online-course content from its previous role analyzing and reviewing a larger variety of supplemental digital-content items, a role it played for more than a decade.


“Along the way, we saw the transition from supplemental resources that were relatively small, meeting maybe 15 or 20 standards, to monsters—we called them tanks—meeting hundreds of standards,” Mr. Bridges said. “That was the big ‘aha!’ moment for us where we saw that’s what is happening in this industry.”

Yet despite their differences, the two groups were among the first such organizations with the foresight to see the coming influence of online courses, and they became effective partners in leading the creation of the second version of iNACOL’s national standards for such courses. The 46-page document, issued in October of last year, includes 52 standards on which to grade courses across the categories of content, instructional design, student assessment, technology, and course evaluation and support.

Those standards have served as one of two templates to consider in CLRN’s review process for online courses. In evaluating courses, the three outside teachers who are hired for a Saturday at a time to act as reviewers also now grade the courses against any of the applicable Common Core State Standards. Reviews of English and math courses, at the pace of roughly four a month total, began last October, with similar reviews of science, social studies, and foreign-language online courses having begun this spring, Mr. Bridges said.

Several other states have followed California and Texas into online-course review based on the iNACOL standards or some variation on them. Ohio enacted legislation in 2010 to establish its own distance-learning clearinghouse that began with the review of approved Advanced Placement courses for all students in Ohio public schools. Georgia and Idaho launched similar clearinghouses this year. Washington state also has for several years had an agency similar in structure to the TXVSN, Mr. Bridges said.

Critical Questions

Still, about 30 percent of CLRN’s website traffic comes from out-of-state users wanting to access the organization’s public reviews, a sign Mr. Bridges said suggests states that follow the examples of CLRN and the TXVSN are in a minority. And iNACOL’s chief operating officer, Matthew Wicks, said even states that have taken those steps are mostly measuring inputs, or dimensions inherent in the course’s composition, rather than outcomes, or measures of a course’s effectiveness.

“We feel like the quality standards produced in the past are a good starting point, … but they themselves don’t guarantee quality,” said Mr. Wicks, who added that there is growing pressure to focus on outcomes. “This has become a hotter topic, in part because of increased activity in online learning, and in part because of the reports that have been critical of full-time online learning programs.”


Some critics of virtual education have suggested that for-profit providers have worked to stop legislators from enacting policies that would more accurately measure the quality of online-course outcomes because those measures would not look good for some providers who are trying to offer a course at the lowest possible cost to the company. Gene V. Glass, a senior researcher for the National Education Policy Center in Boulder, Colo., suggests most outcome-related accountability measures for online courses are vulnerable to abuse and manipulation, such as the use of computer login time as a measurement in states with seat-time requirements, or the administration of unproctored and unsupervised exams. For-profit providers are opposed to measures that are less vulnerable to abuse because they usually need more human involvement and cost more, said Mr. Glass, who is one of several researchers associated with the NEPC who have issued recent reports criticizing for-profit online learning.

“All of those moves have costs associated with them, and the cost of providing courses is [otherwise] so low,” he said. “You can see why the providers don’t want any part of that stuff. They’re going to fight it tooth and nail. “But public and media pressure is prompting some states to take a deeper look at how to measure the outcomes of online learning effectively. While Arizona Gov. Jan Brewer, a Republican, in May vetoed a bill that would have included several new accountability measures for online courses, including requiring live proctors for exams, Colorado Gov. John Hickenlooper, a Democrat, signed a measure the same month that commissioned a study to explore how his state’s online learning courses can be more effective.

Part of the study will focus on what measures could be implemented to more accurately measure the effectiveness of those online courses and programs, as well as the progress of a population that often includes a high number of mobile students and students who enter an online course well behind grade level.

A version of this article appeared in the August 29, 2012 edition of Education Week as Virtual Ed. Faces Challenges In Building Quality Controls


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
IT Management Webinar
Build a Digitally Responsive Educational Organization for Effective Digital-Age Learning
Chart a guided pathway to digital agility and build support for your organization’s mission and vision through dialogue and collaboration.
Content provided by Bluum
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Data Webinar
Drive Instruction With Mastery-Based Assessment
Deliver the right data at the right time—in the right format—and empower better decisions.
Content provided by Instructure
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Teaching Profession Webinar
How Does Educator Well-Being Impact Social-Emotional Awareness in Schools?
Explore how adult well-being is key to promoting healthy social-emotional behaviors for students. Get strategies to reduce teacher stress.
Content provided by International Baccalaureate

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Classroom Technology Case Study: The Hard Transition to 1-to-1 Computing Continues
Eighty-five percent of educators said their districts now have a device for each individual student at all grade levels.
6 min read
Illustration of young girl using laptop.
F. Sheehan for Education Week/Getty
Classroom Technology Opinion 4 Strategies to Help Students Manage Cellphone Use in School
Rules only get you so far; they don't teach young people how to manage challenges.
Tom Harrison
3 min read
Images shows a stylized artistic landscape with soothing colors.
Classroom Technology 5 Ways to Use Technology to Improve Teaching and Learning
Here's an outline of the priorities schools need to address, with links to helpful resources.
2 min read
Arial illustration of a diverse group of kids sitting in a circle with their teacher and surrounded by laptops and zoom windows all around them.
Daniel Hertzberg for Education Week
Classroom Technology Letter to the Editor Personal Cellphones Are Not Instructional Tools
An educator makes the argument that personal devices have no place in instruction in this letter to the editor.
1 min read
Illustration of an open laptop receiving an email.