Corrected: A previous version of this story misstated the grade level PARCC is targeting in the division and multiplication example 54÷9=24÷6. It is grade 3.
As teachers begin shaping lessons for the common standards, many are wondering how to prepare their students for tests that won’t be ready for at least two years. But sample items being drafted for those exams offer early ideas of what lies ahead.
Two large groups of states are using federal Race to the Top money to create new suites of exams for the Common Core State Standards. Those consortia have recently begun work with private vendors to develop items—questions and tasks—for the tests. But each group has produced a range of sample test items to help those vendors get an idea of what the states want, and experts say they offer valuable insight into the tests that are expected to emerge in 2014-15.
“What we are starting to see here are tests that really get at a deeper understanding on the part of students, not just superficial knowledge,” said Robert L. Linn, an assessment expert and professor emeritus of education at the University of Colorado at Boulder who reviewed a sampling of the consortia’s materials. “But unless students are really prepared for them, it’s going to be a huge challenge.”
Mr. Linn predicted that even with sample items to guide them, vendors will find it tough to develop tasks and questions that fully reflect the aims of the two state groups, the Smarter Balanced Assessment Consortium and the Partnership for Assessment of Readiness for Colleges and Careers, or PARCC.
“Where the real difficulty comes up is when you actually develop the items,” he said. “It will be a challenge for vendors to come up with items that meet these specifications. They are used to writing items for state tests that do not get at this depth of knowledge.”
This item, from the Smarter Balanced Assessment Consortium, illustrates one of many examples of how two state testing consortia are trying to guide vendors as they design tests for the common standards.
SOURCE: Adapted from Smarter Balanced Assessment Consortium
Sample items for the English/language arts exams, crafted by work groups of the Smarter Balanced consortium in conjunction with test-makers ETS and Measured Progress, offer a glimpse of what that group has in mind. A few were presented at a meeting of the American Federation of Teachers in Baltimore this summer, and others have been circulating among states for feedback.
One selected-response item asks 5th graders to read an article about how scientists track bird migration and to identify the two paragraphs that contain the author’s opinions on the topic. The question taps key skills required in the common standards, such as comprehending “content rich” nonfiction and citing textual evidence for an argument.
A constructed-response item for 11th graders asks them to read excerpts from an 1872 speech by women’s rights activist Susan B. Anthony and the “Second Treatise of Civil Government” by English philosopher John Locke, published in 1690. They must identify the ideas common to both pieces and discuss how Locke’s ideas support Anthony’s arguments, citing evidence from each to support their interpretations.
One of the aspects of the consortia’s work that represents perhaps the greatest departure from current state testing practice is the inclusion of performance tasks, which engage students in more complex, prolonged exercises. Smarter Balanced’s policy director, Susan Gendron, a former commissioner of education in Maine, offered an example of that approach, too, at the Baltimore meeting.
The sample task, scheduled to take 105 minutes, asks 6th graders to read an interview with a teenager who started a charity to help Peruvian orphans. It directs them to articles and vidoes on specified Web pages to learn more about other young people who devote themselves to helping those in need.
The students answer constructed-response questions that require them to describe what they’ve learned, analyze the meanings of key words, and discuss how they evaluated the reliability of their Web resources. They must research and present a five-minute speech about a “young wonder” of their choice, complete with audiovisual representations.
The fact that all students will now be asked to dive into complex performance tasks is a marked departure from current practice, according to Kris Kaase, who oversaw state assessments in Mississippi from 2002 to 2010.
“These are very high-level kinds of tasks students are going to be asked to do,” he said. “It is significant that all kids are being asked to do this sort of thing. Some students might have been tested in this way, but not all students.”
No Guesswork
Sample math items being circulated by the consortia attempt to plumb not only students’ procedural skills, but their conceptual understanding. Item specifications produced by Smarter Balanced, for instance, include, among other things, an alternative to a typical way of gauging elementary students’ grasp of fractions.
The more traditional approach offers students four graphical depictions of the fraction two-fifths and asks which is correct. The other approach offers the same four graphic models, but asks students to say whether each one has two-fifths of its whole shaded. The latter, according to the consortium’s notes, makes it harder to guess a right answer and requires an understanding of the different forms that fraction can take.
“To perform well on these kinds of assessment items, just having good test-taking skills will not be enough of an edge to perform well,” said Mr. Kaase, who now runs a Jackson-based consulting company that works with states and districts on testing, curriculum, and accountability issues.
“One of the questions is, can we make stronger claims [about students’ skills] from a differently designed item,” he said. “Because of the way these items are being constructed, it seems likely we would be able to make those stronger claims.”
Building Fluency
A sample math performance task by Smarter Balanced asks 6th graders to figure out what they need to build a community garden to a given set of specifications for $450.
During two test sessions totaling up to two hours, students would have to calculate many figures, including the perimeter, surface areas, and volume of each section of the garden, and make a sketch based on their calculations. They must figure out how much wood and soil are needed and how many tomato and carrot plants to buy, given their cost, the garden’s size, and each plant’s need for space. Finally, they must show how their project will stay within its allotted budget.
Materials developed by PARCC, too, illustrate for vendors item types that require a grasp of the topic, said Mr. Kaase. One, for instance, asks 4th graders to plot the following numbers along a number line: 2, 5/4, 3x1/2, 3/4+3/4, and 2-1/10.
“You have to understand the meaning of the numbers and how they relate in order to answer this,” Mr. Kaase said.
Chuck Pack, the chairman of the math department at Tahlequah High School in Oklahoma, which belongs to the PARCC consortium, said he is pleased with the sample items he sees being developed by the consortium.
“They made me hopeful,” said Mr. Pack, who is a teacher-leader for PARCC, helping colleagues deepen their knowledge of the group’s work. “I was a little concerned at first blush, because they’re really complex. But they’re good math problems. They’re above the level of what we’re currently doing, but they’re attainable.”
He pointed to one illustrative example in PARCC’s materials that tries to gauge students’ fluency in division and multiplication. It offers five equations, such as 54÷9=24÷6, and asks 3rd graders to specify whether each is true or false.
“I like that it does multiple assessments in one item,” he said. “It asks kids to work each of those problems easily and be comfortable with it, which is what fluency is.”
PARCC expects to release sample items in English/language arts and math later this month, including prototypes developed under contract with the Institute for Learning at the University of Pittsburgh and the Charles A. Dana Center at the University of Texas at Austin.
Time Crunch
The state consortia have big challenges on their hands, however, and time is among the biggest, experts said. Mr. Kaase recalled that when he was Mississippi’s assessment director, it took at least three years to produce a new version of a state test.
In the case of the common assessments, he said, “we’re not talking about a new version of an existing assessment. We’re talking about a revolutionary kind of change,” not only with different kinds of items, but also computer-based or computer-adaptive technology. “What they are trying to do with these items pushes the bounds,” he said. “To get this done in the amount of time they have is going to be a challenge.”
Many milestones lie ahead before the consortia can deliver fully populated banks of test items. In the coming months, both groups will conduct sessions in which items are tried out with students and their feedback is obtained. Smarter Balanced will train teachers as item writers and has prepared a bank of training materials to assist in that work. Both consortia will conduct trials next year before full-fledged field tests in spring 2014.
Even as sample items are crafted to help guide vendors on item-writing, the consortia and their partners caution that the item-development process is lengthy and full of revisions.
In a presentation to the National Assessment Governing Board in Washington this month, Jeffrey Nellhaus, PARCC’s assessment director, said he was acutely aware that the “field is hungry to see” how the goals of the common standards will be “made manifest” in assessment items, and is eager to examine the prototype items the consortium will get from the two research universities.
“It’s a real challenge to assess the common core in assessment items,” he said. “We’ve been working with national experts. ... But it may take some iterations in our development process.”
As officials from the Dana Center cautioned in an overview of the PARCC project, “Prototyping is for learning, and it can be messy.”
