Crafting a New Generation of Assessments

By Katie Ash — February 17, 2009 5 min read
  • Save to favorites
  • Print

The power of technology to improve assessments, in part by providing useful data on not just what students know, but also on how they arrive at their answers, has been left largely untapped, especially in science, concludes a paper released today.

Computerized testing can be less expensive and deliver faster results—two advantages that many school districts have benefitted from—but it also has the potential to provide richer data on student performance, which could help inform curriculum and instruction, according to the report by the Washington-based think tank Education Sector.

“We should be thinking about how we design the assessments to fit more with cognitive science—to really think about how students progress in their learning so we can not only get better evidence about how they’re doing but get better evidence for us to get the ‘why,’ ” Bill Tucker, the chief operating officer of Education Sector and the author of the report, said in an interview.

See Also

Listen to a podcast on “Beyond the Bubble” from Education Sector.

But Scott Marion, the associate director of the Dover, N.H.-based Center for Assessment—a nonprofit organization that works with states and districts to improve their testing and accountability systems—de-emphasizes the importance of technology to improve assessment and encourages states not to wait until the technology is put into place to start reformatting testing models.

“We could have a lot better assessments without going near computers, and we could have computer-based assessments that are the same lousy assessments we have now,” he says. “I think it’s fair to join [technology and assessment improvement], but my only fear is that people think, ‘We can’t go to the future of assessment until we can provide the [technological] infrastructure in every classroom.’ ”

Perhaps a better starting place for states without a strong technological infrastructure in place, says Tucker, is to “really build that infrastructure anticipating that as we go, we’re going to think about new ways to use that infrastructure.” By building flexibility into the system, states can leave open the potential to explore more innovative assessments later, he suggests.

Promising Models

Although little progress has been made in harnessing the potential technology has to craft performance-based assessments, which aim to simulate complex real-world experiences, and measure students’ approaches to them, there are a handful of promising models, according to the paper.

One is the Problem-Solving in Technology-Rich Environments, or TRE, project. Created in 2003 with a test group of 2,000 students, TRE assesses scientific-inquiry skills by presenting students with a problem—such as questions about buoyancy, volume, and mass—to be answered in a performance-based model, rather than by multiple-choice questions. In this case, students gathered information by running experiments with a simulated helium balloon.

The computer simulation program then recorded data on what the students clicked on, how long they spent on each task, and what methods they tried in order to arrive at their answers.

Educators could then examine how students answered the questions and award partial points based on their approach to the problem, as well as see more detailed data on what students’ thought processes and methodologies were compared with what a single multiple-choice question provides.

Other examples, such as the River City project—a computerized simulation game created by Harvard researchers—as well as the Educational Testing Service’s Cognitively-Based Assessment, which is being piloted in schools in Portland, Maine, represent promising models. None of them currently has widespread implementation, according to the Education Sector paper.

Bridging the Gap

To encourage more states to embrace innovative assessment models, the federal government should provide incentives to states or districts that put them into place, the report recommends.

“That’s the role that states can play,” says Tucker. “The federal government can really develop the research, spur these tools, change the way we’re doing assessment procurement right now, but the states are really the place where it gets implemented or tested.”

To bridge the gap between what technology is capable of and the assessments currently given to students, the report calls for a second, smaller fund in addition to the current federal dollars for assessment that goes toward “the research and development of the next generation of assessment technology and practice.”

And those new assessment models should be open to the public for evaluation and criticism, says the report.

“That’s the way that we as a public are going to be able to check up to see if these tools are working,” Tucker says. “If we can think about it early on and make sure that we structure any sort of funding-incentive grants [to encourage transparency], it’s going to be helpful.”

Given the recent focus on science education, as well as concern about U.S. student’s mixed performance on international and American assessments in that subject, that sector provides a good jumping-off point for these policy recommendations, Tucker says.

“If you look at the research and what the recommendations are for science instruction, it’s a place where you really do need both content and process skills together,” he says. “It’s absolutely necessary to understand science content, but it’s also absolutely necessary to think like a scientist.”

In addition, “things like simulation and digital tools—that’s the way we’re doing science right now,” says Tucker. “It’s not divorced from the field.”

Another reason why science could be a good place to start, says Tucker, as well as Marion from the Center for Assessment, is because assessments in science don’t count toward schools’ adequate yearly progress under the No Child Left Behind Act. As a result, states and districts may have more flexibility to try different kinds of assessments.

“It’s still a place that might be a little more fertile,” says Tucker.

The report is the second in a series of Education Sector’s Next Generation of Accountability initiative, following a paper published last November called “Measuring Skills for the 21st Century.”


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Classroom Technology Webinar
Seamless Integrations for Engagement in the Classroom
Learn how to seamlessly integrate new technologies into your classroom to support student engagement. 
Content provided by GoGuardian
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Recruitment & Retention Webinar
Be the Change: Strategies to Make Year-Round Hiring Happen
Learn how to leverage actionable insights to diversify your recruiting efforts and successfully deploy a year-round recruiting plan.
Content provided by Frontline
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Equity & Diversity Webinar
Critical Ways Leaders Can Build a Culture of Belonging and Achievement
Explore innovative practices for using technology to build an environment of belonging and achievement for all staff and students.
Content provided by DreamBox Learning

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment Latest Round of Federal Grants Aims to Make States' Assessments More Equitable, Precise
The U.S. Department of Education awarded over $29 million in competitive grants to 10 state education agencies.
2 min read
Assessment review data 599911460
Assessment Opinion Are There Better Ways Than Standardized Tests to Assess Students? Educators Think So
Student portfolios and school community surveys are but two of the many alternatives to standardized tests.
3 min read
Illustration of students in virus environment facing wave of test sheets.
Collage by Vanessa Solis/Education Week (Images: iStock/DigitalVision Vectors/Getty)
Assessment Letter to the Editor We Need NAEP
The president and CEO of Knowledge Alliance responds to a recent opinion essay's criticism of the National Assessment of Educational Progress.
1 min read
Illustration of an open laptop receiving an email.
Assessment Letter to the Editor 2022 Assessment ‘Most Important’ Ever
The executive director of the National Assessment Governing Board responds to criticism of NAEP in this letter to the editor.
1 min read
Illustration of an open laptop receiving an email.