Assessment

Vendors at Odds Over Nevada Testing Problems

By Michele Molnar & Andrew Ujifusa — May 04, 2015 6 min read

When breakdowns disrupted Nevada’s online testing last month, state officials were left searching for answers—and they blamed a vendor charged with administering the test, as well as the consortium of states that designed the common-core-aligned exams.

But the testing provider accused of botching the assessments, Measured Progress, has said the responsibility lies at least partly with another vendor, the American Institutes for Research, which it claims was late in delivering critical software needed to make sure the exams could be administered smoothly.

The dust-up is the latest of the battles that have erupted in a number of states, including California and New Mexico, over common-core testing contracts.

But Nevada’s woes underscore another reality, at least for states belonging to the Smarter Balanced Assessment Consortium: In some cases, they’re counting on different testing vendors—who may be rivals—with different responsibilities to help them carry out ambitious new online exams under tight deadlines.

Smarter Balanced Testing Vendors

The following companies have testing administration contracts with 18 states.

State Vendor
California ETS
Connecticut AIR
Delaware AIR
Idaho AIR
Hawaii AIR
Maine AIR
Michigan DRC
Missouri CTB/McGraw Hill
Montana Measured Progress
Nevada Measured Progress
New Hampshire AIR
North Dakota Measured Progress
Oregon AIR
South Dakota AIR
Vermont AIR
Washington AIR
West Virginia AIR
Wisconsin ETS

SOURCE: Smarter Balanced Assessment Consortium

“I’m extremely frustrated at the disruption this has caused in our state. Nevada was ready. Our system was there,” said Steve Canavero, the state’s deputy superintendent for student achievement.

The AIR, a Washington-based nonprofit, received approximately $6 million of its $14.3 million contract from Smarter Balanced to develop an “open source” platform for delivering the test in Nevada and other consortium states that wanted to use that technology. But Measured Progress and Nevada officials say the technology was not ready, and Smarter Balanced also says that portions of the code were not delivered to states until earlier this year.

The AIR, however, has defended its performance.

In 2012, Smarter Balanced contracted with the AIR to build a test-delivery system to register students, present the assessment items to them, and deliver the results from the tests to reporting systems. The contract was awarded in collaboration with DRC to modify the AIR’s existing test-delivery system to meet Smarter Balanced specifications.

DRC, based in Maple Grove, Minn., declined to comment for this article.

The consortium’s announcement about the contract award had highlighted the proven platform the AIR would use as the foundation for the open-source software.

“AIR’s system is used by several states to deliver statewide online assessments,” the press release said. “It is compatible with the range of technology in schools, including computers, laptops, and tablets—requiring only a secure Internet browser for administration. The system will be optimized for release in its open-source version in fall 2014.”

The open version would be designed to allow any company or developer to build upon, or enhance, the technology because its code would be publicly available. That was necessary, so different vendors could access the code and use it for test administration.

Smarter Balanced states made independent decisions about which vendor they would contract with to actually administer the assessments. Eleven states chose the AIR. Montana, Nevada, and North Dakota chose Measured Progress, a Dover, N.H.-based assessment company.

Smarter Balanced is one of two major state consortia that devised tests tied to the Common Core State Standards. Most states have begun administering such tests this year.

Signs of Trouble

The initial problems in Nevada happened in mid-March, when the state had to delay the start of Smarter Balanced tests by two weeks because the system did not perform predictably, said Mr. Canavero.

“The code wasn’t functioning like it needed to function,” he said. “The system wasn’t able to be stress-tested at a level where we could feel confident that our students could start a testing session and have a productive testing experience.”

Even after the two-week delay in March, test administration continued to be plagued by the inability of students to log in or to complete their exams. Multiple patches provided by the AIR that the organization said would solve the issue failed to do so, Mr. Canavero said. And when students in Nevada’s Clark County schools—the fifth-largest district in the country, with an enrollment of about 318,000—tried to start the tests, the system also balked, he said.

On April 15, the state had to halt the exams altogether, and after problems continued, it resumed testing only on a limited basis.

Nevada officials sent a letter notifying Measured Progress and Smarter Balanced that the state deemed them to be in “breach of contract.”

Just 7,000 students statewide were able to take the Smarter Balanced exams on April 24, Mr. Canavero said.

In response to the breach-of-contract letter, Measured Progress released its own statement. Without naming names, the company pointed to “multiple and delayed deliveries of the source code” that it said was necessary to ensure that the Nevada assessment could be sufficiently tested so it would perform as expected when large numbers of test-takers were on it.

Of 19 major components required from the AIR for the open-source test delivery platform, five were released in January or later, and one was delivered in November, which was after the Sept. 30, 2014 release of most of the code, according to Smarter Balanced. The consortium indicated in an email that the AIR made some of the source code or precursors to it available months before the final delivery dates.

‘Minimal Delays’

In response to questions about the AIR’s performance, Jon Cohen, the president of its assessment division, said his organization had been charged with delivering an enormously complex system across multiple states, under tight deadlines, and has performed well.

“There have been minimal delays,” Mr. Cohen said in a phone interview. He added that most software products experience delays in the development and release process.

“Our scheduled delivery of the system was in September,” he said. “We got most of it delivered in time and [provided] updates after that.”

Martin Borg, the president and CEO of Measured Progress, said in a statement that his company performed “exhaustive quality-assurance testing every time we received updated code from the Smarter Balanced vendor.” However, he said, the delays in receiving the code from the vendor—he did not identify the AIR by name—limited the ability of Measured Progress to predict how the assessment-delivery system would perform during peak traffic times.

Of the roughly 840,000 Smarter Balanced tests slated to be taken by about 211,000 students in Nevada, between 8 percent and 23 percent of the exams had been completed by last week, with students finishing a greater share of their English/language arts exams than their math tests. Those figures are far off pace for all students in grades 3-8 to finish their Smarter Balanced exams, according to Mr. Canavero.

The state testing window officially closes June 12, but many Nevada districts end their school years in May.

Problems Beyond Nevada

Major controversies about testing have emerged in other states, too.

In Montana, Measured Progress increased the computer-server capacity after issues arose with delivery of Smarter Balanced tests there. On April 15, Superintendent of Public Instruction Denise Juneau announced that the common-core-aligned tests would be voluntary because of a delay in opening the testing window.

In California, Pearson, an education company based in the United Kingdom, wants state officials to re-bid the testing contract that was awarded to the Lawrenceville, N.J.-based Educational Testing Service. Pearson contends the contract, which could be worth a quarter of a billion dollars, was “invalid” and “illegal,” on the grounds that Pearson was the lowest bidder, public records were destroyed in the procurement process, and one of its proposed strategies wound up in the winning ETS proposal.

In New Mexico, meanwhile, the AIR is suing the state to stop a contract award to Pearson for work associated with the Partnership for Assessment of Readiness for College and Careers, or PARCC, the other main consortium developing common-core tests. The AIR is arguing that the proposal for the work unfairly favored Pearson because it was bundled with work the company had already done for PARCC. The AIR estimates the maximum value of the contract to be $1 billion.

A version of this article appeared in the May 06, 2015 edition of Education Week as Nevada Exams Hit Tech Trouble

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
A Safe Return to Schools is Possible with Testing
We are edging closer to a nationwide return to in-person learning in the fall. However, vaccinations alone will not get us through this. Young children not being able to vaccinate, the spread of new and
Content provided by BD
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Teaching Webinar
Meeting the Moment: Accelerating Equitable Recovery and Transformative Change
Educators are deciding how best to re-establish routines such as everyday attendance, rebuild the relationships for resilient school communities, and center teaching and learning to consciously prioritize protecting the health and overall well-being of students
Content provided by Campaign for Grade-Level Reading
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Reading & Literacy Webinar
Addressing Learning Loss: What Schools Need to Accelerate Reading Instruction in K-3
When K-3 students return to classrooms this fall, there will be huge gaps in foundational reading skills. Does your school or district need a plan to address learning loss and accelerate student growth? In this
Content provided by PDX Reading

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment Standardized Tests Could Be in Jeopardy in Wake of Biden Decisions, Experts Say
Has the Biden administration shored up statewide tests this year only to risk undermining long-term public backing for them?
6 min read
Illustration of students in virus environment facing wave of test sheets.
Collage by Vanessa Solis/Education Week (Images: iStock/DigitalVision Vectors/Getty)
Assessment How Can Teachers Better Understand Students? A New Breed of Assessment Will Try to Help
Researchers will work to create formative assessments that can give teachers a window into students’ emerging identities and strengths
4 min read
In this Nov. 19, 2020, file photo, sixth-grade students listen to instruction in class at Cape Cod Lighthouse Charter School in East Harwich, Mass.
Researchers hope to create new assessments to help teachers gain deeper insights into the identities and strengths of their students, like these 6th graders at Cape Cod Lighthouse Charter School in East Harwich, Mass.
Elise Amendola/AP
Assessment Opinion It's Time We Begin Using Assessments to Look Forward, Instead of Back
Schools do not get much value from high-stakes tests. Many are now allowing schools to use better assessments to guide student learning.
Seth Feldman
5 min read
shutterstock 19525837
Shutterstock
Assessment Opinion Grading Has Always Been an Imperfect Exercise. COVID-19 Made It Worse
It’s hard reducing the complexity of each student’s social, emotional, and academic learning to a letter grade. Maybe we’re doing it wrong.
Lory Walker Peroff
4 min read
A student's grades are unknown
Robert Neubecker for Education Week