Assessment

Vendors at Odds Over Nevada Testing Problems

By Michele Molnar & Andrew Ujifusa — May 04, 2015 6 min read

When breakdowns disrupted Nevada’s online testing last month, state officials were left searching for answers—and they blamed a vendor charged with administering the test, as well as the consortium of states that designed the common-core-aligned exams.

But the testing provider accused of botching the assessments, Measured Progress, has said the responsibility lies at least partly with another vendor, the American Institutes for Research, which it claims was late in delivering critical software needed to make sure the exams could be administered smoothly.

The dust-up is the latest of the battles that have erupted in a number of states, including California and New Mexico, over common-core testing contracts.

But Nevada’s woes underscore another reality, at least for states belonging to the Smarter Balanced Assessment Consortium: In some cases, they’re counting on different testing vendors—who may be rivals—with different responsibilities to help them carry out ambitious new online exams under tight deadlines.

Smarter Balanced Testing Vendors

The following companies have testing administration contracts with 18 states.

State Vendor
California ETS
Connecticut AIR
Delaware AIR
Idaho AIR
Hawaii AIR
Maine AIR
Michigan DRC
Missouri CTB/McGraw Hill
Montana Measured Progress
Nevada Measured Progress
New Hampshire AIR
North Dakota Measured Progress
Oregon AIR
South Dakota AIR
Vermont AIR
Washington AIR
West Virginia AIR
Wisconsin ETS

SOURCE: Smarter Balanced Assessment Consortium

“I’m extremely frustrated at the disruption this has caused in our state. Nevada was ready. Our system was there,” said Steve Canavero, the state’s deputy superintendent for student achievement.

The AIR, a Washington-based nonprofit, received approximately $6 million of its $14.3 million contract from Smarter Balanced to develop an “open source” platform for delivering the test in Nevada and other consortium states that wanted to use that technology. But Measured Progress and Nevada officials say the technology was not ready, and Smarter Balanced also says that portions of the code were not delivered to states until earlier this year.

The AIR, however, has defended its performance.

In 2012, Smarter Balanced contracted with the AIR to build a test-delivery system to register students, present the assessment items to them, and deliver the results from the tests to reporting systems. The contract was awarded in collaboration with DRC to modify the AIR’s existing test-delivery system to meet Smarter Balanced specifications.

DRC, based in Maple Grove, Minn., declined to comment for this article.

The consortium’s announcement about the contract award had highlighted the proven platform the AIR would use as the foundation for the open-source software.

“AIR’s system is used by several states to deliver statewide online assessments,” the press release said. “It is compatible with the range of technology in schools, including computers, laptops, and tablets—requiring only a secure Internet browser for administration. The system will be optimized for release in its open-source version in fall 2014.”

The open version would be designed to allow any company or developer to build upon, or enhance, the technology because its code would be publicly available. That was necessary, so different vendors could access the code and use it for test administration.

Smarter Balanced states made independent decisions about which vendor they would contract with to actually administer the assessments. Eleven states chose the AIR. Montana, Nevada, and North Dakota chose Measured Progress, a Dover, N.H.-based assessment company.

Smarter Balanced is one of two major state consortia that devised tests tied to the Common Core State Standards. Most states have begun administering such tests this year.

Signs of Trouble

The initial problems in Nevada happened in mid-March, when the state had to delay the start of Smarter Balanced tests by two weeks because the system did not perform predictably, said Mr. Canavero.

“The code wasn’t functioning like it needed to function,” he said. “The system wasn’t able to be stress-tested at a level where we could feel confident that our students could start a testing session and have a productive testing experience.”

Even after the two-week delay in March, test administration continued to be plagued by the inability of students to log in or to complete their exams. Multiple patches provided by the AIR that the organization said would solve the issue failed to do so, Mr. Canavero said. And when students in Nevada’s Clark County schools—the fifth-largest district in the country, with an enrollment of about 318,000—tried to start the tests, the system also balked, he said.

On April 15, the state had to halt the exams altogether, and after problems continued, it resumed testing only on a limited basis.

Nevada officials sent a letter notifying Measured Progress and Smarter Balanced that the state deemed them to be in “breach of contract.”

Just 7,000 students statewide were able to take the Smarter Balanced exams on April 24, Mr. Canavero said.

In response to the breach-of-contract letter, Measured Progress released its own statement. Without naming names, the company pointed to “multiple and delayed deliveries of the source code” that it said was necessary to ensure that the Nevada assessment could be sufficiently tested so it would perform as expected when large numbers of test-takers were on it.

Of 19 major components required from the AIR for the open-source test delivery platform, five were released in January or later, and one was delivered in November, which was after the Sept. 30, 2014 release of most of the code, according to Smarter Balanced. The consortium indicated in an email that the AIR made some of the source code or precursors to it available months before the final delivery dates.

‘Minimal Delays’

In response to questions about the AIR’s performance, Jon Cohen, the president of its assessment division, said his organization had been charged with delivering an enormously complex system across multiple states, under tight deadlines, and has performed well.

“There have been minimal delays,” Mr. Cohen said in a phone interview. He added that most software products experience delays in the development and release process.

“Our scheduled delivery of the system was in September,” he said. “We got most of it delivered in time and [provided] updates after that.”

Martin Borg, the president and CEO of Measured Progress, said in a statement that his company performed “exhaustive quality-assurance testing every time we received updated code from the Smarter Balanced vendor.” However, he said, the delays in receiving the code from the vendor—he did not identify the AIR by name—limited the ability of Measured Progress to predict how the assessment-delivery system would perform during peak traffic times.

Of the roughly 840,000 Smarter Balanced tests slated to be taken by about 211,000 students in Nevada, between 8 percent and 23 percent of the exams had been completed by last week, with students finishing a greater share of their English/language arts exams than their math tests. Those figures are far off pace for all students in grades 3-8 to finish their Smarter Balanced exams, according to Mr. Canavero.

The state testing window officially closes June 12, but many Nevada districts end their school years in May.

Problems Beyond Nevada

Major controversies about testing have emerged in other states, too.

In Montana, Measured Progress increased the computer-server capacity after issues arose with delivery of Smarter Balanced tests there. On April 15, Superintendent of Public Instruction Denise Juneau announced that the common-core-aligned tests would be voluntary because of a delay in opening the testing window.

In California, Pearson, an education company based in the United Kingdom, wants state officials to re-bid the testing contract that was awarded to the Lawrenceville, N.J.-based Educational Testing Service. Pearson contends the contract, which could be worth a quarter of a billion dollars, was “invalid” and “illegal,” on the grounds that Pearson was the lowest bidder, public records were destroyed in the procurement process, and one of its proposed strategies wound up in the winning ETS proposal.

In New Mexico, meanwhile, the AIR is suing the state to stop a contract award to Pearson for work associated with the Partnership for Assessment of Readiness for College and Careers, or PARCC, the other main consortium developing common-core tests. The AIR is arguing that the proposal for the work unfairly favored Pearson because it was bundled with work the company had already done for PARCC. The AIR estimates the maximum value of the contract to be $1 billion.

A version of this article appeared in the May 06, 2015 edition of Education Week as Nevada Exams Hit Tech Trouble

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
How Districts Are Centering Relationships and Systemic SEL for Back to School 21-22
As educators and leaders consider how SEL fits into their reopening and back-to-school plans, it must go beyond an SEL curriculum. SEL is part of who we are as educators and students, as well as
Content provided by Panorama Education
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Achievement Webinar
The Fall K-3 Classroom: What the data imply about composition, challenges and opportunities
The data tracking learning loss among the nation’s schoolchildren confirms that things are bad and getting worse. The data also tells another story — one with serious implications for the hoped for learning recovery initiatives
Content provided by Campaign for Grade-Level Reading
Student Well-Being Online Summit Student Mental Health
Attend this summit to learn what the data tells us about student mental health, what schools can do, and best practices to support students.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment Opinion Alternatives to Standardized Tests During a Pandemic Year
Three educators suggest alternatives to federally mandated standardized testing during this year undercut by COVID-19.
7 min read
Images shows colorful speech bubbles that say "Q," "&," and "A."
iStock/Getty
Assessment Opinion AP Exams Can't Be Business as Usual This Year
The College Board seems unconcerned with the collateral damage of its pandemic approach, writes an assistant superintendent of curriculum and instruction.
Pete Bavis
5 min read
Illustration of large boat in turbulent waters with other smaller boats falling into the abyss.
iStock/Getty Images Plus
Assessment Federal Lawmakers Urge Miguel Cardona to Let States Cancel Tests, Highlighting Discord
A letter from Democratic members to the new education secretary calls for an end to the "flawed" system of annual standardized exams.
3 min read
Jamaal Bowman speaks to reporters after voting at a polling station inside Yonkers Middle/High School in Yonkers, N.Y. on June 23, 2020.
Jamaal Bowman speaks to reporters after voting at a polling station inside Yonkers Middle/High School in Yonkers, N.Y. on June 23, 2020.
John Minchillo/AP
Assessment How Two Years of Pandemic Disruption Could Shake Up the Debate Over Standardized Testing
Moves to opt out of state tests and change how they're given threaten to reignite fights over high-stakes assessments.
9 min read
Image of a student at a desk.
patat/iStock/Getty