Assessment

Vendors at Odds Over Nevada Testing Problems

By Michele Molnar & Andrew Ujifusa — May 04, 2015 6 min read
  • Save to favorites
  • Print

When breakdowns disrupted Nevada’s online testing last month, state officials were left searching for answers—and they blamed a vendor charged with administering the test, as well as the consortium of states that designed the common-core-aligned exams.

But the testing provider accused of botching the assessments, Measured Progress, has said the responsibility lies at least partly with another vendor, the American Institutes for Research, which it claims was late in delivering critical software needed to make sure the exams could be administered smoothly.

The dust-up is the latest of the battles that have erupted in a number of states, including California and New Mexico, over common-core testing contracts.

But Nevada’s woes underscore another reality, at least for states belonging to the Smarter Balanced Assessment Consortium: In some cases, they’re counting on different testing vendors—who may be rivals—with different responsibilities to help them carry out ambitious new online exams under tight deadlines.

Smarter Balanced Testing Vendors

The following companies have testing administration contracts with 18 states.

State Vendor
California ETS
Connecticut AIR
Delaware AIR
Idaho AIR
Hawaii AIR
Maine AIR
Michigan DRC
Missouri CTB/McGraw Hill
Montana Measured Progress
Nevada Measured Progress
New Hampshire AIR
North Dakota Measured Progress
Oregon AIR
South Dakota AIR
Vermont AIR
Washington AIR
West Virginia AIR
Wisconsin ETS

SOURCE: Smarter Balanced Assessment Consortium

“I’m extremely frustrated at the disruption this has caused in our state. Nevada was ready. Our system was there,” said Steve Canavero, the state’s deputy superintendent for student achievement.

The AIR, a Washington-based nonprofit, received approximately $6 million of its $14.3 million contract from Smarter Balanced to develop an “open source” platform for delivering the test in Nevada and other consortium states that wanted to use that technology. But Measured Progress and Nevada officials say the technology was not ready, and Smarter Balanced also says that portions of the code were not delivered to states until earlier this year.

The AIR, however, has defended its performance.

In 2012, Smarter Balanced contracted with the AIR to build a test-delivery system to register students, present the assessment items to them, and deliver the results from the tests to reporting systems. The contract was awarded in collaboration with DRC to modify the AIR’s existing test-delivery system to meet Smarter Balanced specifications.

DRC, based in Maple Grove, Minn., declined to comment for this article.

The consortium’s announcement about the contract award had highlighted the proven platform the AIR would use as the foundation for the open-source software.

“AIR’s system is used by several states to deliver statewide online assessments,” the press release said. “It is compatible with the range of technology in schools, including computers, laptops, and tablets—requiring only a secure Internet browser for administration. The system will be optimized for release in its open-source version in fall 2014.”

The open version would be designed to allow any company or developer to build upon, or enhance, the technology because its code would be publicly available. That was necessary, so different vendors could access the code and use it for test administration.

Smarter Balanced states made independent decisions about which vendor they would contract with to actually administer the assessments. Eleven states chose the AIR. Montana, Nevada, and North Dakota chose Measured Progress, a Dover, N.H.-based assessment company.

Smarter Balanced is one of two major state consortia that devised tests tied to the Common Core State Standards. Most states have begun administering such tests this year.

Signs of Trouble

The initial problems in Nevada happened in mid-March, when the state had to delay the start of Smarter Balanced tests by two weeks because the system did not perform predictably, said Mr. Canavero.

“The code wasn’t functioning like it needed to function,” he said. “The system wasn’t able to be stress-tested at a level where we could feel confident that our students could start a testing session and have a productive testing experience.”

Even after the two-week delay in March, test administration continued to be plagued by the inability of students to log in or to complete their exams. Multiple patches provided by the AIR that the organization said would solve the issue failed to do so, Mr. Canavero said. And when students in Nevada’s Clark County schools—the fifth-largest district in the country, with an enrollment of about 318,000—tried to start the tests, the system also balked, he said.

On April 15, the state had to halt the exams altogether, and after problems continued, it resumed testing only on a limited basis.

Nevada officials sent a letter notifying Measured Progress and Smarter Balanced that the state deemed them to be in “breach of contract.”

Just 7,000 students statewide were able to take the Smarter Balanced exams on April 24, Mr. Canavero said.

In response to the breach-of-contract letter, Measured Progress released its own statement. Without naming names, the company pointed to “multiple and delayed deliveries of the source code” that it said was necessary to ensure that the Nevada assessment could be sufficiently tested so it would perform as expected when large numbers of test-takers were on it.

Of 19 major components required from the AIR for the open-source test delivery platform, five were released in January or later, and one was delivered in November, which was after the Sept. 30, 2014 release of most of the code, according to Smarter Balanced. The consortium indicated in an email that the AIR made some of the source code or precursors to it available months before the final delivery dates.

‘Minimal Delays’

In response to questions about the AIR’s performance, Jon Cohen, the president of its assessment division, said his organization had been charged with delivering an enormously complex system across multiple states, under tight deadlines, and has performed well.

“There have been minimal delays,” Mr. Cohen said in a phone interview. He added that most software products experience delays in the development and release process.

“Our scheduled delivery of the system was in September,” he said. “We got most of it delivered in time and [provided] updates after that.”

Martin Borg, the president and CEO of Measured Progress, said in a statement that his company performed “exhaustive quality-assurance testing every time we received updated code from the Smarter Balanced vendor.” However, he said, the delays in receiving the code from the vendor—he did not identify the AIR by name—limited the ability of Measured Progress to predict how the assessment-delivery system would perform during peak traffic times.

Of the roughly 840,000 Smarter Balanced tests slated to be taken by about 211,000 students in Nevada, between 8 percent and 23 percent of the exams had been completed by last week, with students finishing a greater share of their English/language arts exams than their math tests. Those figures are far off pace for all students in grades 3-8 to finish their Smarter Balanced exams, according to Mr. Canavero.

The state testing window officially closes June 12, but many Nevada districts end their school years in May.

Problems Beyond Nevada

Major controversies about testing have emerged in other states, too.

In Montana, Measured Progress increased the computer-server capacity after issues arose with delivery of Smarter Balanced tests there. On April 15, Superintendent of Public Instruction Denise Juneau announced that the common-core-aligned tests would be voluntary because of a delay in opening the testing window.

In California, Pearson, an education company based in the United Kingdom, wants state officials to re-bid the testing contract that was awarded to the Lawrenceville, N.J.-based Educational Testing Service. Pearson contends the contract, which could be worth a quarter of a billion dollars, was “invalid” and “illegal,” on the grounds that Pearson was the lowest bidder, public records were destroyed in the procurement process, and one of its proposed strategies wound up in the winning ETS proposal.

In New Mexico, meanwhile, the AIR is suing the state to stop a contract award to Pearson for work associated with the Partnership for Assessment of Readiness for College and Careers, or PARCC, the other main consortium developing common-core tests. The AIR is arguing that the proposal for the work unfairly favored Pearson because it was bundled with work the company had already done for PARCC. The AIR estimates the maximum value of the contract to be $1 billion.

A version of this article appeared in the May 06, 2015 edition of Education Week as Nevada Exams Hit Tech Trouble

Events

Curriculum Webinar Computer Science Education Movement Gathers Momentum. How Should Schools React?
Discover how schools can expand opportunities for students to study computer science education.
School & District Management Webinar Fostering Student Well-Being with Programs That Work
Protecting student well-being has never been more important. Join this webinar to learn how to ensure your programs yield the best outcomes.
Reading & Literacy Webinar 'Science of Reading': What Are the Components?
Learn how to adopt a “science of reading” approach to early literacy to effectively build students’ vocabulary and content knowledge.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment Don’t Use State Tests ‘Punitively,’ Ed. Secretary Cardona Warns
As federal accountability restarts after two years, guidance from the department underscores how complicated that could be.
5 min read
Image of data, target goals, and gaining ground.
iStock/Getty
Assessment Latest Round of Federal Grants Aims to Make States' Assessments More Equitable, Precise
The U.S. Department of Education awarded over $29 million in competitive grants to 10 state education agencies.
2 min read
Assessment review data 599911460
vladwei/iStock/Getty<br/>
Assessment Opinion Are There Better Ways Than Standardized Tests to Assess Students? Educators Think So
Student portfolios and school community surveys are but two of the many alternatives to standardized tests.
3 min read
Illustration of students in virus environment facing wave of test sheets.
Collage by Vanessa Solis/Education Week (Images: iStock/DigitalVision Vectors/Getty)
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Assessment Whitepaper
Empowering personalized instruction with a three-tiered approach to learning evidence
Navvy is the first classroom assessment system designed to empower personalized learning by providing granular, reliable, and proximal le...
Content provided by Pearson