Assessment

Vendors at Odds Over Nevada Testing Problems

By Michele Molnar & Andrew Ujifusa — May 04, 2015 6 min read
  • Save to favorites
  • Print

When breakdowns disrupted Nevada’s online testing last month, state officials were left searching for answers—and they blamed a vendor charged with administering the test, as well as the consortium of states that designed the common-core-aligned exams.

But the testing provider accused of botching the assessments, Measured Progress, has said the responsibility lies at least partly with another vendor, the American Institutes for Research, which it claims was late in delivering critical software needed to make sure the exams could be administered smoothly.

The dust-up is the latest of the battles that have erupted in a number of states, including California and New Mexico, over common-core testing contracts.

But Nevada’s woes underscore another reality, at least for states belonging to the Smarter Balanced Assessment Consortium: In some cases, they’re counting on different testing vendors—who may be rivals—with different responsibilities to help them carry out ambitious new online exams under tight deadlines.

Smarter Balanced Testing Vendors

The following companies have testing administration contracts with 18 states.

State Vendor
California ETS
Connecticut AIR
Delaware AIR
Idaho AIR
Hawaii AIR
Maine AIR
Michigan DRC
Missouri CTB/McGraw Hill
Montana Measured Progress
Nevada Measured Progress
New Hampshire AIR
North Dakota Measured Progress
Oregon AIR
South Dakota AIR
Vermont AIR
Washington AIR
West Virginia AIR
Wisconsin ETS

SOURCE: Smarter Balanced Assessment Consortium

“I’m extremely frustrated at the disruption this has caused in our state. Nevada was ready. Our system was there,” said Steve Canavero, the state’s deputy superintendent for student achievement.

The AIR, a Washington-based nonprofit, received approximately $6 million of its $14.3 million contract from Smarter Balanced to develop an “open source” platform for delivering the test in Nevada and other consortium states that wanted to use that technology. But Measured Progress and Nevada officials say the technology was not ready, and Smarter Balanced also says that portions of the code were not delivered to states until earlier this year.

The AIR, however, has defended its performance.

In 2012, Smarter Balanced contracted with the AIR to build a test-delivery system to register students, present the assessment items to them, and deliver the results from the tests to reporting systems. The contract was awarded in collaboration with DRC to modify the AIR’s existing test-delivery system to meet Smarter Balanced specifications.

DRC, based in Maple Grove, Minn., declined to comment for this article.

The consortium’s announcement about the contract award had highlighted the proven platform the AIR would use as the foundation for the open-source software.

“AIR’s system is used by several states to deliver statewide online assessments,” the press release said. “It is compatible with the range of technology in schools, including computers, laptops, and tablets—requiring only a secure Internet browser for administration. The system will be optimized for release in its open-source version in fall 2014.”

The open version would be designed to allow any company or developer to build upon, or enhance, the technology because its code would be publicly available. That was necessary, so different vendors could access the code and use it for test administration.

Smarter Balanced states made independent decisions about which vendor they would contract with to actually administer the assessments. Eleven states chose the AIR. Montana, Nevada, and North Dakota chose Measured Progress, a Dover, N.H.-based assessment company.

Smarter Balanced is one of two major state consortia that devised tests tied to the Common Core State Standards. Most states have begun administering such tests this year.

Signs of Trouble

The initial problems in Nevada happened in mid-March, when the state had to delay the start of Smarter Balanced tests by two weeks because the system did not perform predictably, said Mr. Canavero.

“The code wasn’t functioning like it needed to function,” he said. “The system wasn’t able to be stress-tested at a level where we could feel confident that our students could start a testing session and have a productive testing experience.”

Even after the two-week delay in March, test administration continued to be plagued by the inability of students to log in or to complete their exams. Multiple patches provided by the AIR that the organization said would solve the issue failed to do so, Mr. Canavero said. And when students in Nevada’s Clark County schools—the fifth-largest district in the country, with an enrollment of about 318,000—tried to start the tests, the system also balked, he said.

On April 15, the state had to halt the exams altogether, and after problems continued, it resumed testing only on a limited basis.

Nevada officials sent a letter notifying Measured Progress and Smarter Balanced that the state deemed them to be in “breach of contract.”

Just 7,000 students statewide were able to take the Smarter Balanced exams on April 24, Mr. Canavero said.

In response to the breach-of-contract letter, Measured Progress released its own statement. Without naming names, the company pointed to “multiple and delayed deliveries of the source code” that it said was necessary to ensure that the Nevada assessment could be sufficiently tested so it would perform as expected when large numbers of test-takers were on it.

Of 19 major components required from the AIR for the open-source test delivery platform, five were released in January or later, and one was delivered in November, which was after the Sept. 30, 2014 release of most of the code, according to Smarter Balanced. The consortium indicated in an email that the AIR made some of the source code or precursors to it available months before the final delivery dates.

‘Minimal Delays’

In response to questions about the AIR’s performance, Jon Cohen, the president of its assessment division, said his organization had been charged with delivering an enormously complex system across multiple states, under tight deadlines, and has performed well.

“There have been minimal delays,” Mr. Cohen said in a phone interview. He added that most software products experience delays in the development and release process.

“Our scheduled delivery of the system was in September,” he said. “We got most of it delivered in time and [provided] updates after that.”

Martin Borg, the president and CEO of Measured Progress, said in a statement that his company performed “exhaustive quality-assurance testing every time we received updated code from the Smarter Balanced vendor.” However, he said, the delays in receiving the code from the vendor—he did not identify the AIR by name—limited the ability of Measured Progress to predict how the assessment-delivery system would perform during peak traffic times.

Of the roughly 840,000 Smarter Balanced tests slated to be taken by about 211,000 students in Nevada, between 8 percent and 23 percent of the exams had been completed by last week, with students finishing a greater share of their English/language arts exams than their math tests. Those figures are far off pace for all students in grades 3-8 to finish their Smarter Balanced exams, according to Mr. Canavero.

The state testing window officially closes June 12, but many Nevada districts end their school years in May.

Problems Beyond Nevada

Major controversies about testing have emerged in other states, too.

In Montana, Measured Progress increased the computer-server capacity after issues arose with delivery of Smarter Balanced tests there. On April 15, Superintendent of Public Instruction Denise Juneau announced that the common-core-aligned tests would be voluntary because of a delay in opening the testing window.

In California, Pearson, an education company based in the United Kingdom, wants state officials to re-bid the testing contract that was awarded to the Lawrenceville, N.J.-based Educational Testing Service. Pearson contends the contract, which could be worth a quarter of a billion dollars, was “invalid” and “illegal,” on the grounds that Pearson was the lowest bidder, public records were destroyed in the procurement process, and one of its proposed strategies wound up in the winning ETS proposal.

In New Mexico, meanwhile, the AIR is suing the state to stop a contract award to Pearson for work associated with the Partnership for Assessment of Readiness for College and Careers, or PARCC, the other main consortium developing common-core tests. The AIR is arguing that the proposal for the work unfairly favored Pearson because it was bundled with work the company had already done for PARCC. The AIR estimates the maximum value of the contract to be $1 billion.

A version of this article appeared in the May 06, 2015 edition of Education Week as Nevada Exams Hit Tech Trouble

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
A Whole Child Approach to Supporting Positive Student Behavior 
To improve student behavior, it’s important to look at the root causes. Social-emotional learning may play a preventative role.

A whole child approach can proactively support positive student behaviors.

Join this webinar to learn how.
Content provided by Panorama
Recruitment & Retention Live Online Discussion A Seat at the Table: Why Retaining Education Leaders of Color Is Key for Student Success
Today, in the United States roughly 53 percent of our public school students are young people of color, while approximately 80 percent of the educators who lead their classrooms, schools, and districts are white. Racial
Jobs January 2022 Virtual Career Fair for Teachers and K-12 Staff
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Assessment Whitepaper
Report: Now is the Time to Reimagine Assessments
In New Meridian’s latest whitepaper, Founder and CEO Arthur VanderVeen identifies five principles to boldly re-imagine assessment and cre...
Content provided by New Meridian
Assessment State Test Results Are In. Are They Useless?
While states, districts, and schools pore over data from spring 2021 tests, experts urge caution over how to interpret and use the results.
9 min read
FILE - In this Jan. 17, 2016 file photo, a sign is seen at the entrance to a hall for a college test preparation class in Bethesda, Md. The $380 million test coaching industry is facing competition from free or low-cost alternatives in what their founders hope will make the process of applying to college more equitable. Such innovations are also raising questions about the relevance and the fairness of relying on standardized tests in admissions process.
A sign is posted at the entrance to a hall for a test-preparation class. Assessment experts say educators should use data from spring 2021 tests with caution.
Alex Brandon/AP
Assessment Data Young Adolescents' Scores Trended to Historic Lows on National Tests. And That's Before COVID Hit
The past decade saw unprecedented declines in the National Assessment of Educational Progress's longitudinal study.
3 min read
Assessment Long a Testing Bastion, Florida Plans to End 'Outdated' Year-End Exams
Florida Gov. Ron DeSantis said the state will shift to "progress monitoring" starting in the 2022-23 school year.
5 min read
Florida Governor Ron DeSantis speaks at the opening of a monoclonal antibody site in Pembroke Pines, Fla., on Aug. 18, 2021.
Florida Gov. Ron DeSantis said he believes a new testing regimen is needed to replace the Florida Standards Assessment, which has been given since 2015.
Marta Lavandier/AP