Crafting a New Generation of Assessments

By Katie Ash — February 17, 2009 5 min read
  • Save to favorites
  • Print

The power of technology to improve assessments, in part by providing useful data on not just what students know, but also on how they arrive at their answers, has been left largely untapped, especially in science, concludes a paper released today.

Computerized testing can be less expensive and deliver faster results—two advantages that many school districts have benefitted from—but it also has the potential to provide richer data on student performance, which could help inform curriculum and instruction, according to the report by the Washington-based think tank Education Sector.

“We should be thinking about how we design the assessments to fit more with cognitive science—to really think about how students progress in their learning so we can not only get better evidence about how they’re doing but get better evidence for us to get the ‘why,’ ” Bill Tucker, the chief operating officer of Education Sector and the author of the report, said in an interview.

See Also

Listen to a podcast on “Beyond the Bubble” from Education Sector.

But Scott Marion, the associate director of the Dover, N.H.-based Center for Assessment—a nonprofit organization that works with states and districts to improve their testing and accountability systems—de-emphasizes the importance of technology to improve assessment and encourages states not to wait until the technology is put into place to start reformatting testing models.

“We could have a lot better assessments without going near computers, and we could have computer-based assessments that are the same lousy assessments we have now,” he says. “I think it’s fair to join [technology and assessment improvement], but my only fear is that people think, ‘We can’t go to the future of assessment until we can provide the [technological] infrastructure in every classroom.’ ”

Perhaps a better starting place for states without a strong technological infrastructure in place, says Tucker, is to “really build that infrastructure anticipating that as we go, we’re going to think about new ways to use that infrastructure.” By building flexibility into the system, states can leave open the potential to explore more innovative assessments later, he suggests.

Promising Models

Although little progress has been made in harnessing the potential technology has to craft performance-based assessments, which aim to simulate complex real-world experiences, and measure students’ approaches to them, there are a handful of promising models, according to the paper.

One is the Problem-Solving in Technology-Rich Environments, or TRE, project. Created in 2003 with a test group of 2,000 students, TRE assesses scientific-inquiry skills by presenting students with a problem—such as questions about buoyancy, volume, and mass—to be answered in a performance-based model, rather than by multiple-choice questions. In this case, students gathered information by running experiments with a simulated helium balloon.

The computer simulation program then recorded data on what the students clicked on, how long they spent on each task, and what methods they tried in order to arrive at their answers.

Educators could then examine how students answered the questions and award partial points based on their approach to the problem, as well as see more detailed data on what students’ thought processes and methodologies were compared with what a single multiple-choice question provides.

Other examples, such as the River City project—a computerized simulation game created by Harvard researchers—as well as the Educational Testing Service’s Cognitively-Based Assessment, which is being piloted in schools in Portland, Maine, represent promising models. None of them currently has widespread implementation, according to the Education Sector paper.

Bridging the Gap

To encourage more states to embrace innovative assessment models, the federal government should provide incentives to states or districts that put them into place, the report recommends.

“That’s the role that states can play,” says Tucker. “The federal government can really develop the research, spur these tools, change the way we’re doing assessment procurement right now, but the states are really the place where it gets implemented or tested.”

To bridge the gap between what technology is capable of and the assessments currently given to students, the report calls for a second, smaller fund in addition to the current federal dollars for assessment that goes toward “the research and development of the next generation of assessment technology and practice.”

And those new assessment models should be open to the public for evaluation and criticism, says the report.

“That’s the way that we as a public are going to be able to check up to see if these tools are working,” Tucker says. “If we can think about it early on and make sure that we structure any sort of funding-incentive grants [to encourage transparency], it’s going to be helpful.”

Given the recent focus on science education, as well as concern about U.S. student’s mixed performance on international and American assessments in that subject, that sector provides a good jumping-off point for these policy recommendations, Tucker says.

“If you look at the research and what the recommendations are for science instruction, it’s a place where you really do need both content and process skills together,” he says. “It’s absolutely necessary to understand science content, but it’s also absolutely necessary to think like a scientist.”

In addition, “things like simulation and digital tools—that’s the way we’re doing science right now,” says Tucker. “It’s not divorced from the field.”

Another reason why science could be a good place to start, says Tucker, as well as Marion from the Center for Assessment, is because assessments in science don’t count toward schools’ adequate yearly progress under the No Child Left Behind Act. As a result, states and districts may have more flexibility to try different kinds of assessments.

“It’s still a place that might be a little more fertile,” says Tucker.

The report is the second in a series of Education Sector’s Next Generation of Accountability initiative, following a paper published last November called “Measuring Skills for the 21st Century.”


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Assessment Webinar
The State of Assessment in K-12 Education
What is the impact of assessment on K-12 education? What does that mean for administrators, teachers and most importantly—students?
Content provided by Instructure
Jobs January 2022 Virtual Career Fair for Teachers and K-12 Staff
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Reading & Literacy Webinar
Proven Strategies to Improve Reading Scores
In this webinar, education and reading expert Stacy Hurst will provide a look at some of the biggest issues facing curriculum coordinators, administrators, and teachers working in reading education today. You will: Learn how schools
Content provided by Reading Horizons

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment Data Young Adolescents' Scores Trended to Historic Lows on National Tests. And That's Before COVID Hit
The past decade saw unprecedented declines in the National Assessment of Educational Progress's longitudinal study.
3 min read
Assessment Long a Testing Bastion, Florida Plans to End 'Outdated' Year-End Exams
Florida Gov. Ron DeSantis said the state will shift to "progress monitoring" starting in the 2022-23 school year.
5 min read
Florida Governor Ron DeSantis speaks at the opening of a monoclonal antibody site in Pembroke Pines, Fla., on Aug. 18, 2021.
Florida Gov. Ron DeSantis said he believes a new testing regimen is needed to replace the Florida Standards Assessment, which has been given since 2015.
Marta Lavandier/AP
Assessment Spotlight Spotlight on Assessment in 2021
In this Spotlight, review newest assessment scores, see how districts will catch up with their supports for disabled students, plus more.
Assessment 'Nation's Report Card' Has a New Reading Framework, After a Drawn-Out Battle Over Equity
The new framework for the National Assessment of Educational Progress will guide development of the 2026 reading test.
10 min read
results 925693186 02