Assessment

Adaptive Testing Gains Momentum, Prompts Concerns

By Benjamin Herold — July 09, 2013 7 min read

The federal government and dozens of states are slowly paving the way for widespread use of high-stakes online exams that adjust the difficulty of their questions based on the skill levels of individual test-takers.

But dueling congressional proposals to overhaul the Elementary and Secondary Education Act, currently known as No Child Left Behind, highlight lingering disagreements over how computer-adaptive assessment should function and have reignited long-standing concerns held by advocates for students with disabilities.

At the heart of the debate is the extent to which computer-adaptive exams should ask questions that are above or below a test-taker’s grade level. Skeptics worry that too much leeway, along with a relaxation of the federal requirement that state tests cover the full range of grade-level content, could result in dumbed-down assessments—and eventually dumbed-down instruction—for struggling students.

“Prior to NCLB, states were allowed to give fifth grade students a third grade test and call them proficient,” said Laura Kaloi, who co-chairs the education task force for the Consortium for Citizens with Disabilities, a Washington-based advocacy group that has 1,115 member organizations. “We don’t want computer-adaptive testing to go down a similar path.”

Full Potential

In the U.S. House of Representatives, a proposal to overhaul the ESEA would explicitly allow off-grade-level test items. Rep. Tom Petri, R-Wis., who sponsored an earlier bill that was folded into that broader legislation, said that approach would allow states “to use these new assessments to their full potential” by precisely pinpointing what students know and are able to do.

A competing Senate proposal to rewrite the law includes a requirement to assess students on grade level, but Ms. Kaloi said it leaves open a “loophole” that “bad actors” could exploit to deliver tests that are heavily off grade level.

Broadly speaking, the benefits of computer-adaptive testing are widely agreed upon: shorter, more secure exams; more precise information on what students know; and faster turnaround of test results for use by educators, administrators, and policymakers.

But the stakes surrounding the contentious details could be high.

Beginning in 2014-15, more than 20 states intend to administer high-stakes computer-adaptive tests to millions of children. Those exams, which are being created by the Smarter Balanced Assessment Consortium and will be tied to the Common Core State Standards, are expected to include a limited number of off-grade-level questions.

The U.S. Department of Education is funding the Smarter Balanced effort to the tune of $175 million, but the group’s exams will still need federal approval before they can be used.

Ms. Kaloi said disability-rights advocates are still in “wait-and-see mode” regarding Smarter Balanced’s approach.

In its present form, the ESEA neither prohibits nor favors any specific type of test—computer-adaptive or otherwise. But it does require students to be tested on the full depth and breadth of on-grade-level content, which has made it difficult to win federal approval to use some adaptive exams for accountability purposes.

New Breed of Tests

How It Works: Adaptive Testing

Learn how adaptive testing works in this 2012 video and read the related story, Adaptive Testing Evolves to Assess Common-Core Skills.

In the next version of the act, lawmakers from both parties and houses of Congress want the new breed of assessments officially acknowledged to ensure they can be used by states.

“I think I’ve sort of won these battles on recognizing the need to do computer-adaptive testing,” said Sen. Al Franken, D-Minn., who has pushed proposals on the issue since 2011.

Mr. Franken said the fixed-form NCLB-era assessments do not provide accurate measures of what struggling and advanced students know, creating a “race to the middle.”

Computer-adaptive assessments, on the other hand, rely on complex algorithms to feed students questions targeted to their individual skill levels based on their prior responses. The more questions a student gets right, the harder the subsequent questions will be.

A current Senate proposal sponsored by Sen. Tom Harkin of Iowa, a Democrat, and supported by Mr. Franken, to reauthorize the ESEA calls for all exams used for federal accountability purposes to assess students in two ways: whether they are performing at grade level or not, and the specific grade level at which they are performing.

Mr. Franken supports the inclusion of test questions that are above and below grade level, though proponents of the Senate legislation stressed that the bill requires all exams used for federal accountability purposes to measure the full range of grade-level standards so that lower-performing students are not held to lower expectations.

Ms. Kaloi said the Consortium for Citizens with Disabilities supports the overall Senate proposal sponsored by Mr. Harkin to revise the ESEA, but is disappointed with the bill’s language around assessment.

“We remain concerned about the loophole that could allow out-of-level testing,” she said, referring to the bill’s specific allowance to test for students’ grade-level performance, which Ms. Kaloi said could be open to interpretation and abuse.

Disability-rights groups do not support the House Republican proposal to overhaul the law, which Rep. Petri’s office said would also require states to measure students’ actual grade-level performance, as well as whether they are on grade level or not.

Ms. Kaloi pointed to exams created by the American Institutes for Research, which has been contracted by six states to provide computer-adaptive assessments, as an example of how the tests should protect struggling students.

On the Washington-based institutes’ 5th grade exams, for example, reading passages might vary from student to student according to the complexity of the language and concepts they contain. But all test-takers would be asked to engage in grade-appropriate higher-order thinking skills, such as making inferences about what characters in the passage might do next.

“It’s important that computer-adaptive testing be standards-based, meaning there is a blueprint that ensures every kid sees test questions that reflect the full breadth and depth of on-grade-level content,” said Jon Cohen, AIR’s executive vice president and director of assessment.

Delaware, Hawaii, and Oregon have all won federal approval for those types of adaptive exams and have been successfully using them for years, said Mr. Cohen. His organization’s approach, he said, helps ensure that struggling students are not limited to being assessed—or taught—only basic skills, such as recall of facts or rote applications of formulas.

Mr. Cohen decried a “big push” from lobbyists on behalf of “companies that sell adaptive tests that are not tied to a strong testing blueprint” to allow off-grade-level testing and tests that do not cover the full range of grade-level content.

“I think that would be a step backwards,” he said.

The biggest test of the viability of high-stakes computer-adaptive assessment will come in 2014-15, when many of the schools covered by the Smarter Balanced Assessment Consortium could struggle with the technical hurdles of administering hundreds of exams online at once.

Challenges Ahead

Figuring out an effective, equitable way to implement the exams’ adaptive functions will also be a challenge, said Joe Willhoft, the consortium’s executive director.

“We want to be very careful about our use of out-of-grade level items for students who are struggling,” he said. “If we do use [them], we would restrain ourselves to only one or two times” per test.

Such limits constitute a trade-off, Mr. Willhoft acknowledged.

“That approach doesn’t take as much advantage of the effectiveness of computer-adaptive testing as possible,” he said. “But it does adhere to the value that all students are assessed according to content standards.”

Smarter Balanced is still developing its bank of test items. A field test of the fully adaptive exams, in which roughly 20 percent of the students represented by the consortium will participate, will take place next spring.

Ms. Kaloi said that disability-rights groups won’t be able to effectively evaluate Smarter Balanced’s adaptive exams until closer to that time.

Mr. Willhoft said he did not anticipate any problems for states seeking Education Department approval to use Smarter Balanced exams to meet their accountability requirements.

“By virtue of its grant award, the department has clearly approved the overall design,” he said.

A spokesman for the department offered a different take.

“The Smarter Balanced Assessment Consortium exams, just like any other state test, will have to go through a new peer review if they will be used for accountability purposes,” said Daren Briscoe. “Being funded for the project does not obviate the need for the review.”

Coverage of entrepreneurship and innovation in education and school design is supported in part by a grant from the Carnegie Corporation of New York. Education Week retains sole editorial control over the content of this coverage.
A version of this article appeared in the July 11, 2013 edition of Education Week as Adaptive Testing Gains Momentum, Prompts Worries

Events

School & District Management Live Event Education Week Leadership Symposium
Education Week's Premier Leadership Event for K12 School & District Leaders.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Law & Courts Webinar
The Future of Criminal Justice Reform: A Sphere Education Initiative Conversation
America’s criminal justice system is in crisis and calls for reform are dominating the national debate. Join Cato’s Sphere Education Initiative and Education Week for a webinar on criminal justice and policing featuring the nation’s
Content provided by Cato Institute
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
Equity, Care and Connection: New SEL Tools and Practices to Support Students and Adults
As school districts plan to welcome students back into buildings for the upcoming school year, this is the perfect time to take a hard look at both our practices and our systems to build a
Content provided by Panorama Education

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment Opinion What Federally Mandated State Tests Are Good For (And What They Aren’t)
Spring 2021 testing is happening. That can be a good thing—if the goal is about more than school accountability.
Stuart Kahl
5 min read
Two people analyze test data
Visual Generation/iStock/Getty
Assessment Opinion The National Assessment Governing Board’s Troubling Gag Order
NAGB's recently released restrictions on how its board members can communicate set a troubling precedent.
3 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Assessment Whitepaper
INVALSI Addresses Italy’s COVID-19 Learning Loss
Find out how INVALSI worked with TAO to develop a plan of action that can serve as a model for other education leaders grappling with the...
Content provided by TAO by Open Assessment Technologies
Assessment Biden Administration's Level of Tolerance for Cutting Standardized Tests Comes Into Focus
A distinction has grown between states having to make tests available, and districts deciding it's not practical to make students take them.
8 min read
Image of a test sheet.
sengchoy/iStock/Getty