Special Report
Classroom Technology

Quality Control a Challenge for Virtual Ed.

By Ian Quillen — August 27, 2012 6 min read

As online learning has entered the mainstream—with roughly a third of the nation’s high school students enrolled in at least one online course, according to a report released in June 2011—more states have created policies, procedures, and even organizations for evaluating the quality of such courses and other online content available to students.

But instituting those quality-control measures is not without challenges.

For starters, the creation of such systems often requires legislation, which must be crafted by lawmakers who aren’t always familiar with online learning’s nuances and may not be eager to launch new programs in tight budget times.

Further, the peculiarities of individual states’ education policies—especially on standards and materials approval—mean the kind of oversight that might work effectively in one state might not function well in another.

“There’s no state we could look at and say they’ve got it all figured out,” said John Watson, the founder of the Evergreen Education Group, the Durango, Colo.-based research firm that puts out the annual “Keeping Pace” reports measuring growth and changes in K-12 online learning. Taking a functional system in one state and applying it to the policy realities of another would be “fairly difficult,” Mr. Watson added.

Even the California Learning Resource Network, or CLRN, and the Texas Virtual School Network, or TXVSN, which partnered to write a second edition of the iNACOL National Standards for Quality Online Courses, released last October, have considerably different authority, responsibilities, and procedures. (The Vienna, Va.-based International Association for K-12 Online Learning, or iNACOL, is a leading advocacy group for proponents of online and blended learning for students.)

The Houston-based TXVSN, for example, must approve all courses offered to regular public or charter school students in Texas, whether the classes are a supplement to brick-and-mortar coursework or part of a full-time virtual program.

By contrast, the Modesto-based CLRN acts in a more advisory role to districts and charter schools seeking to ensure that they purchase high-quality content, said CLRN Director Brian Bridges. The organization can’t deny districts or charters the right to use their online courses of choice. However, CLRN was set this month to announce a new partnership with the University of California system in which only CLRN-approved online courses will be accepted as valid when state universities evaluate high school students’ transcripts.

Further, school districts and state universities are the main creators of the content reviewed by the TXVSN, while CLRN can review content created by districts, universities, or private companies. And unlike CLRN, the Texas group takes ownership of the courses it reviews and acts as the vendor for interested districts or charter schools.

Even the organizations’ roots are different; legislators in 2007 created the TXVSN explicitly for reviewing and approving online-course content. By contrast, the California group has morphed into an organization focused primarily on reviewing online-course content from its previous role analyzing and reviewing a larger variety of supplemental digital-content items, a role it played for more than a decade.

BRIC ARCHIVE

“Along the way, we saw the transition from supplemental resources that were relatively small, meeting maybe 15 or 20 standards, to monsters—we called them tanks—meeting hundreds of standards,” Mr. Bridges said. “That was the big ‘aha!’ moment for us where we saw that’s what is happening in this industry.”

Yet despite their differences, the two groups were among the first such organizations with the foresight to see the coming influence of online courses, and they became effective partners in leading the creation of the second version of iNACOL’s national standards for such courses. The 46-page document, issued in October of last year, includes 52 standards on which to grade courses across the categories of content, instructional design, student assessment, technology, and course evaluation and support.

Those standards have served as one of two templates to consider in CLRN’s review process for online courses. In evaluating courses, the three outside teachers who are hired for a Saturday at a time to act as reviewers also now grade the courses against any of the applicable Common Core State Standards. Reviews of English and math courses, at the pace of roughly four a month total, began last October, with similar reviews of science, social studies, and foreign-language online courses having begun this spring, Mr. Bridges said.

Several other states have followed California and Texas into online-course review based on the iNACOL standards or some variation on them. Ohio enacted legislation in 2010 to establish its own distance-learning clearinghouse that began with the review of approved Advanced Placement courses for all students in Ohio public schools. Georgia and Idaho launched similar clearinghouses this year. Washington state also has for several years had an agency similar in structure to the TXVSN, Mr. Bridges said.

Critical Questions

Still, about 30 percent of CLRN’s website traffic comes from out-of-state users wanting to access the organization’s public reviews, a sign Mr. Bridges said suggests states that follow the examples of CLRN and the TXVSN are in a minority. And iNACOL’s chief operating officer, Matthew Wicks, said even states that have taken those steps are mostly measuring inputs, or dimensions inherent in the course’s composition, rather than outcomes, or measures of a course’s effectiveness.

“We feel like the quality standards produced in the past are a good starting point, … but they themselves don’t guarantee quality,” said Mr. Wicks, who added that there is growing pressure to focus on outcomes. “This has become a hotter topic, in part because of increased activity in online learning, and in part because of the reports that have been critical of full-time online learning programs.”

BRIC ARCHIVE

Some critics of virtual education have suggested that for-profit providers have worked to stop legislators from enacting policies that would more accurately measure the quality of online-course outcomes because those measures would not look good for some providers who are trying to offer a course at the lowest possible cost to the company. Gene V. Glass, a senior researcher for the National Education Policy Center in Boulder, Colo., suggests most outcome-related accountability measures for online courses are vulnerable to abuse and manipulation, such as the use of computer login time as a measurement in states with seat-time requirements, or the administration of unproctored and unsupervised exams. For-profit providers are opposed to measures that are less vulnerable to abuse because they usually need more human involvement and cost more, said Mr. Glass, who is one of several researchers associated with the NEPC who have issued recent reports criticizing for-profit online learning.

“All of those moves have costs associated with them, and the cost of providing courses is [otherwise] so low,” he said. “You can see why the providers don’t want any part of that stuff. They’re going to fight it tooth and nail. “But public and media pressure is prompting some states to take a deeper look at how to measure the outcomes of online learning effectively. While Arizona Gov. Jan Brewer, a Republican, in May vetoed a bill that would have included several new accountability measures for online courses, including requiring live proctors for exams, Colorado Gov. John Hickenlooper, a Democrat, signed a measure the same month that commissioned a study to explore how his state’s online learning courses can be more effective.

Part of the study will focus on what measures could be implemented to more accurately measure the effectiveness of those online courses and programs, as well as the progress of a population that often includes a high number of mobile students and students who enter an online course well behind grade level.

A version of this article appeared in the August 29, 2012 edition of Education Week as Virtual Ed. Faces Challenges In Building Quality Controls

Events

Jobs The EdWeek Top School Jobs Virtual Career Fair
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
Embracing Student Engagement: The Pathway to Post-Pandemic Learning
As schools emerge from remote learning, educators are understandably worried about content and skills that students would otherwise have learned under normal circumstances. This raises the very real possibility that children will face endless hours
Content provided by Newsela
Teaching Live Online Discussion How to Develop Powerful Project-Based Learning
How do you prepare students to be engaged, active, and empowered young adults? Creating a classroom atmosphere that encourages students to pursue critical inquiry and the many skills it requires demands artful planning on the

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Classroom Technology Quiz Quiz: How Much Do You Know About Creating Effective Hybrid Learning Environments?
Quiz Yourself: How effective are your hybrid learning environments?
Classroom Technology Like Police Officers, Educators Have Been Caught on Camera Behaving Badly
Several recent incidents show how students and parents can use their cellphones to capture abusive or racist behavior by some educators.
9 min read
cellphone video 1242161726 02
Petro Bevz/iStock/Getty
Classroom Technology School District Leaders Are Still Worried About Home Internet Access for Students
Schools have scaled-up their efforts to help more kids get online, according to a new survey, but concerns remain about tech equity.
2 min read
Veronica Esquivel, 10, finishes her homework after her virtual school hours while her brother Isias Esquivel sits in front of the computer, Wednesday, Feb. 10, 2021, at their residence in Chicago's predominantly Hispanic Pilsen neighborhood.
Veronica Esquivel, 10, finishes her homework in her Chicago home after a day of virtual school in February.
Shafkat Anowar/AP
Classroom Technology Teachers: Here Are Tips for Using Your New 'Geeky' Skills to Improve Classroom Management
How educators can use the lessons of the pandemic to reshape classroom management for next school year and beyond.
6 min read
A group of people manage a complicated problem
Vanessa Solis/Education Week and iStock/Getty Images Plus