Assessment

Online State Testing in 2018: Mostly Smooth, With One Glaring Exception

By Benjamin Herold — May 08, 2018 7 min read

Just about everything that could go wrong did go wrong with online state testing this year in Tennessee.

First, login problems prevented some students from accessing the exams. The following day, an apparent cyberattack on the state’s assessment vendor prevented thousands more students from submitting their tests.

Then, thanks to human error at some schools, about 1,400 students ended up taking the wrong version of the TNReady exam. And a rogue dump truck severed one of the state’s main fiber-optic cables, causing temporary connectivity problems during the testing period.

Bringing frustrations throughout the state to a boil: It’s the second time in three years that schools have experienced major online-testing disruptions.

“We’re sick and tired of the failures,” said state Rep. Eddie Smith, a Republican.

Along with fellow lawmakers, Smith responded by quickly passing a new state law that prevents scores on this year’s TNReady exam from being used to punish schools, teachers, or students.

The move highlights the significant policy implications when online testing goes awry. It also puts the U.S. Department of Education in an uncomfortable position. Secretary of Education Betsy DeVos could be forced to decide whether to grant Tennessee a waiver from the testing and accountability requirements of the federal Every Student Succeeds Act, setting a precedent the department would likely prefer to avoid.

Online Testing Troubles in 5 States

Across the country, the transition to administering state standardized tests via computer has at times been rocky. Schools have dealt with cyberattacks and poor connectivity, unfamiliar interfaces and login problems, scoring errors and students getting booted from online-testing platforms. In recent years, however, such troubles have generally been more episodic than systemic. During spring 2018, publicly reported online-testing problems occurred in:

New York

Like Tennessee, the New York state education department contracts with a company called Questar.

On April 11, the company was hit with an apparent cyberattack, causing an “unacceptable failure” in New York, said state board of regents Chancellor Betty A. Rosa and state education Commissioner MaryEllen Elia in a statement.

Across the state, students had difficulty logging in and connecting to the testing platform. According to Questar officials, 4,723 New York students were initially unable to submit their test responses as a result of the troubles.

Minnesota

Minor connectivity issues and login problems delayed testing at three Minnetonka schools late last month.

The state’s assessment vendor is global education company Pearson. In a statement, the company said its support teams responded to “sporadic reports of connectivity issues at schools” and “quickly added additional capacity, which immediately mitigated the issue.”

Mississippi

Another state using Questar, Mississippi also experienced problems with intermittent connectivity and some students being unable to submit online tests this year.

All told, 47 of 144 districts were affected, and a total of 606 students in the state experienced problems during a one-hour disruption, according to a spokeswoman for the state education department.

Ohio

Students across the Buckeye State had trouble logging in to state tests on April 18. The issue was resolved later that day. State education officials declined to detail how many students were affected.

Ohio contracts with the American Institutes for Research to administer its statewide exams.

Tennessee

For the second time in three years, Tennessee experienced widespread troubles with its online TNReady exams. The issues stemmed from a number of separate problems and prompted new state legislation.

—Benjamin Herold

Despite the meltdown in Tennessee and smaller issues in a handful of other states, assessment experts say the nation’s K-12 schools mostly seem to be getting the hang of how to administer online tests to millions of students each year.

“I don’t know that we’ll ever get the problems down to zero,” said Marianne Perie, the director of the Center for Assessment and Accountability Research Design at the University of Kansas.

“But paper-and-pencil testing was never error-free, either,” said Perie, who also chairs the technical-advisory committees around online testing in New York and Tennessee.

A Troubled History

Like many states, Tennessee made the decision in 2014 to switch to online tests.

The move was originally spurred by the state’s membership in the Partnership for the Assessment of Readiness for College and Careers, which was preparing to roll out new assessments aligned to the Common Core State Standards.

The Tennessee legislature quickly backed out of PARCC, however, setting in motion a multiyear chain reaction.

In spring 2014, faced with a narrow window to find a new online-testing vendor, the state education department rushed into a $108 million contract with a company called Measurement Inc.

Despite what department officials now describe as concerns about the company’s lack of experience administering a large-scale statewide online test, the state education department pushed forward with a wholesale switch to the new online assessments in 2016.

The administration of the exams that year quickly went awry.

State education Commissioner Candace McQueen responded by suspending testing in grade’s 3-8 and canceling the state’s contract with Measurement Inc.

At the time, Henry Scherich, the company’s president, told Education Week that Tennessee was giving up too quickly on the online tests.

Scherich declined to comment last week, citing ongoing litigation between his company and the Tennessee education department.

By summer 2016, the state had selected Questar as its new assessment vendor. Back then, state officials touted the company’s experience and track record as reasons for awarding it a $30 million per-year contract.

Limited online testing last spring and fall—part of McQueen’s new phased-in approach—was mostly smooth.

This year was supposed to be the breakthrough. All told, 650,000 students were expected to take the TNReady exam. More than 300,000 of them were expected to do so online.

But problems started almost immediately.

A software flaw caused “unacceptable login delays” for some students on the first day of testing, according to a presentation later delivered by Tennessee education officials to state lawmakers.

On the second day of testing, Questar was flooded with unanticipated traffic that overwhelmed the company’s servers and prevented some students from connecting to the TNReady testing platform.

All told, the problems resulted in 5,066 Tennessee students being initially unable to submit their tests, according to Questar. An undetermined number of additional students also experienced login difficulties, and some may have been kicked off the testing platform after they started taking the exam.

In the presentation to the Tennessee legislature, McQueen described her department as “completely devastated” by the disruptions.

Outsiders said Tennessee’s recurring problems point to some kind of deeper underlying trouble.

“I’m not aware of a state that has had a more troubled transition” to online testing, said Douglas A. Levin of the consulting group EdTech Strategies.

‘The Million-Dollar Question’

Questar officials noted that they have successfully administered roughly 3 million tests across multiple states this spring.

But any problems are unacceptable, Brad Baumgartner, the company’s chief operating officer, said in an interview.

The initial login problems were the result of “a design decision we made a year ago, which in hindsight we would not have made,” Baumgartner said.

And the flood of unanticipated traffic on day two of the exams bore the hallmarks of a type of cybersecurity incident known as a distributed denial-of-service, or DDoS, attack. In addition to Tennessee, students in New York and Mississippi were also affected.

Given past DDoS attacks during online state assessments, Questar likely should have been better prepared, said Levin. It’s particularly concerning that the company was apparently running multiple state assessments through the same “load balancing” server, creating a single point of failure when the traffic surge occurred.

Why weren’t such problems caught before the live administration of the exams?

“That’s the million-dollar question,” Baumgartner said.

Questar’s contract with the Tennessee education department is up for renewal this fall. Sara Gast, the communications director for the department, said officials anticipate “revisiting” the state’s deal with the company.

In the meantime, Tennessee educators and lawmakers are out of patience.

“Student and teacher morale has been hurt, parents are increasingly upset, and the credibility of the entire testing system has been severely damaged,” said Jim Wrye, a spokesman for the Tennessee Education Association, the state’s largest teachers’ union.

As the testing problems mounted, TEA members flooded state lawmakers with complaints.

The state legislature responded, quickly passing measures designed to prevent 2017-18 TNReady results from resulting in adverse consequences for those in the K-12 system.

Now, none of this year’s test scores can be used:

• To assign a letter grade to a school under the state’s accountability system or to designate a school as low performing;

• To move a school into the state-run Achievement School District; and

• To make decisions about hiring, firing, or compensating any teacher.

In addition, Tennessee districts were given discretion to determine the extent to which this year’s test scores should be used to determine students’ final grades.

“What we’re doing is finding the best way to hold everyone harmless, but to not have unintended consequences by doing more than we should,” said Smith, the state representative.

What’s Next?

Even as state testing continues, the Tennessee education department has referred the apparent cyberattack to the Davidson County district attorney and investigators from both the state and an undisclosed third party.

In addition, the department takes questions about the validity of this year’s TNReady results “very seriously,” Gast said.

“We have provided guidance on how a school can nullify the results for a student if they feel the interruptions prevented him or her from demonstrating their full understanding of the standards,” she said.

Department staff members are also helping schools recover interrupted test sessions, and they will be conducting numerous analyses to make sure that any unusual score results are excluded.

As for applying for an ESSA waiver, Gast said the department is “still discussing with the U.S. Department of Education what we may need to submit to them,” but remains confident it will ultimately be in compliance with the federal education law.

The reality is that Tennessee’s online-testing mess has left everyone in a difficult position, said Chad Aldeman, a principal at Bellwether Education Partners, a consulting organization.

“The state has not [made] stability a key priority in their testing vendors,” Aldeman said, and as a result,Tennessee may need to be “bailed out by the feds.”

Assistant Editor Alyson Klein contributed reporting to this article.
A version of this article appeared in the May 09, 2018 edition of Education Week as Tenn. Struggles to Clean Up Huge Online Testing Mess

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Recruitment & Retention Webinar
Recruiting and Retaining a More Diverse Teaching Workforce
We discuss the importance of workforce diversity and learn strategies to recruit and retain teachers from diverse backgrounds.
Content provided by EdWeek Top School Jobs
Student Well-Being Webinar Boosting Teacher and Student Motivation During the Pandemic: What It Takes
Join Alyson Klein and her expert guests for practical tips and discussion on how to keep students and teachers motivated as the pandemic drags on.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
A Holistic Approach to Social-Emotional Learning
Register to learn about the components and benefits of holistically implemented SEL.
Content provided by Committee for Children

EdWeek Top School Jobs

Principal
Meredith, New Hampshire
Inter-Lakes School District
Elementary Principal
Washington State
Wenatchee School District
Principal
Meredith, New Hampshire
Inter-Lakes School District
Elementary Principal
Washington State
Wenatchee School District

Read Next

Assessment Biden's Testing Stance Leaves States Tough Choices. Some May Still Try to Avoid Exams
Whether to give tests in person this spring or even test students next school year instead, education leaders confront a complex path.
Flags decorate a space outside the secretary's office at the U.S. Department of Education in Washington.
Flags decorate a space outside the secretary's office at the U.S. Department of Education in Washington.
Jacquelyn Martin/AP
Assessment States Still Must Give Standardized Tests This Year, Biden Administration Announces
But the administration says it would allow states to give tests in the summer or use partial exams due to challenges related to COVID-19.
3 min read
Image of students taking a test.
smolaw11/iStock/Getty
Assessment Timing of Food Stamps Can Affect Students' Test Scores, Study Finds
Hungry students don't test as well, say researchers who found a link between food stamp disbursements and students' exam scores.
5 min read
A sign advertises a program that allows food stamp recipients to use their EBT cards to shop at a farmer's market in Topsham, Maine on March 17, 2017.
Food stamps can be used in some farmers' markets, as at this one in Topsham, Maine. New research shows a link between timing of the aid and student performance on key tests.
Robert F. Bukaty/AP
Assessment New Mexico Asks to Skip Student Testing Again This Year
State officials are seeking permission from federal officials to waive standardized testing for another year, citing the pandemic.
3 min read