Crafting a New Generation of Assessments

By Katie Ash — February 17, 2009 5 min read

The power of technology to improve assessments, in part by providing useful data on not just what students know, but also on how they arrive at their answers, has been left largely untapped, especially in science, concludes a paper released today.

Computerized testing can be less expensive and deliver faster results—two advantages that many school districts have benefitted from—but it also has the potential to provide richer data on student performance, which could help inform curriculum and instruction, according to the report by the Washington-based think tank Education Sector.

“We should be thinking about how we design the assessments to fit more with cognitive science—to really think about how students progress in their learning so we can not only get better evidence about how they’re doing but get better evidence for us to get the ‘why,’ ” Bill Tucker, the chief operating officer of Education Sector and the author of the report, said in an interview.

See Also

Listen to a podcast on “Beyond the Bubble” from Education Sector.

But Scott Marion, the associate director of the Dover, N.H.-based Center for Assessment—a nonprofit organization that works with states and districts to improve their testing and accountability systems—de-emphasizes the importance of technology to improve assessment and encourages states not to wait until the technology is put into place to start reformatting testing models.

“We could have a lot better assessments without going near computers, and we could have computer-based assessments that are the same lousy assessments we have now,” he says. “I think it’s fair to join [technology and assessment improvement], but my only fear is that people think, ‘We can’t go to the future of assessment until we can provide the [technological] infrastructure in every classroom.’ ”

Perhaps a better starting place for states without a strong technological infrastructure in place, says Tucker, is to “really build that infrastructure anticipating that as we go, we’re going to think about new ways to use that infrastructure.” By building flexibility into the system, states can leave open the potential to explore more innovative assessments later, he suggests.

Promising Models

Although little progress has been made in harnessing the potential technology has to craft performance-based assessments, which aim to simulate complex real-world experiences, and measure students’ approaches to them, there are a handful of promising models, according to the paper.

One is the Problem-Solving in Technology-Rich Environments, or TRE, project. Created in 2003 with a test group of 2,000 students, TRE assesses scientific-inquiry skills by presenting students with a problem—such as questions about buoyancy, volume, and mass—to be answered in a performance-based model, rather than by multiple-choice questions. In this case, students gathered information by running experiments with a simulated helium balloon.

The computer simulation program then recorded data on what the students clicked on, how long they spent on each task, and what methods they tried in order to arrive at their answers.

Educators could then examine how students answered the questions and award partial points based on their approach to the problem, as well as see more detailed data on what students’ thought processes and methodologies were compared with what a single multiple-choice question provides.

Other examples, such as the River City project—a computerized simulation game created by Harvard researchers—as well as the Educational Testing Service’s Cognitively-Based Assessment, which is being piloted in schools in Portland, Maine, represent promising models. None of them currently has widespread implementation, according to the Education Sector paper.

Bridging the Gap

To encourage more states to embrace innovative assessment models, the federal government should provide incentives to states or districts that put them into place, the report recommends.

“That’s the role that states can play,” says Tucker. “The federal government can really develop the research, spur these tools, change the way we’re doing assessment procurement right now, but the states are really the place where it gets implemented or tested.”

To bridge the gap between what technology is capable of and the assessments currently given to students, the report calls for a second, smaller fund in addition to the current federal dollars for assessment that goes toward “the research and development of the next generation of assessment technology and practice.”

And those new assessment models should be open to the public for evaluation and criticism, says the report.

“That’s the way that we as a public are going to be able to check up to see if these tools are working,” Tucker says. “If we can think about it early on and make sure that we structure any sort of funding-incentive grants [to encourage transparency], it’s going to be helpful.”

Given the recent focus on science education, as well as concern about U.S. student’s mixed performance on international and American assessments in that subject, that sector provides a good jumping-off point for these policy recommendations, Tucker says.

“If you look at the research and what the recommendations are for science instruction, it’s a place where you really do need both content and process skills together,” he says. “It’s absolutely necessary to understand science content, but it’s also absolutely necessary to think like a scientist.”

In addition, “things like simulation and digital tools—that’s the way we’re doing science right now,” says Tucker. “It’s not divorced from the field.”

Another reason why science could be a good place to start, says Tucker, as well as Marion from the Center for Assessment, is because assessments in science don’t count toward schools’ adequate yearly progress under the No Child Left Behind Act. As a result, states and districts may have more flexibility to try different kinds of assessments.

“It’s still a place that might be a little more fertile,” says Tucker.

The report is the second in a series of Education Sector’s Next Generation of Accountability initiative, following a paper published last November called “Measuring Skills for the 21st Century.”


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Law & Courts Webinar
The Future of Criminal Justice Reform: A Sphere Education Initiative Conversation
America’s criminal justice system is in crisis and calls for reform are dominating the national debate. Join Cato’s Sphere Education Initiative and Education Week for a webinar on criminal justice and policing featuring the nation’s
Content provided by Cato Institute
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Student Well-Being Webinar
Equity, Care and Connection: New SEL Tools and Practices to Support Students and Adults
As school districts plan to welcome students back into buildings for the upcoming school year, this is the perfect time to take a hard look at both our practices and our systems to build a
Content provided by Panorama Education
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Classroom Technology Webinar
Here to Stay – Pandemic Lessons for EdTech in Future Development
What technology is needed in a post pandemic district? Learn how changes in education will impact development of new technologies.
Content provided by AWS

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment Opinion The National Assessment Governing Board’s Troubling Gag Order
NAGB's recently released restrictions on how its board members can communicate set a troubling precedent.
3 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Assessment Whitepaper
INVALSI Addresses Italy’s COVID-19 Learning Loss
Find out how INVALSI worked with TAO to develop a plan of action that can serve as a model for other education leaders grappling with the...
Content provided by TAO by Open Assessment Technologies
Assessment Biden Administration's Level of Tolerance for Cutting Standardized Tests Comes Into Focus
A distinction has grown between states having to make tests available, and districts deciding it's not practical to make students take them.
8 min read
Image of a test sheet.
Assessment Opinion Alternatives to Standardized Tests During a Pandemic Year
Three educators suggest alternatives to federally mandated standardized testing during this year undercut by COVID-19.
7 min read
Images shows colorful speech bubbles that say "Q," "&," and "A."