[Correction: This blog post should have said that the NAEP math items examined in the Brookings Institution study were, on average, two to three years below the 8th grade math recommended by the common-core standards. The algebra items were at about the 6th grade level. The items from the “numbers” strand were at about the 5th grade level.]
A new analysis from the Brookings Institution raises questions anew about what the advent of common standards—and the development of common assessments to complement them—means for the future of NAEP, often called “America’s report card.” Overall, the report suggests that the coming common exams mean “a new era is dawning for NAEP,” though what that future will look like remains murky.
The analysis by Brookings senior fellow Tom Loveless seeks to match up the common standards in mathematics with publicly released test items from the National Assessment of Educational Progress. In algebra, it found that the NAEP items were, on average, two to three years below the 8th grade math recommended by the common-core standards. The study examines NAEP math items from the 8th grade assessment, coding all publicly released items from the algebra and number strands based on the grade at which the common standards recommends teaching the math assessed by the item. In all, 171 items were available.
In an interview, Loveless told me: “You’ll have a state like California participating in the common core, and it will have a state score and another based on NAEP. And because these tests are designed so differently and cover different material, they may or may not say the same thing.”
He added: “There’s the potential that we’re going to confuse a lot of people.”
(For an earlier Education Week look at what the common standards could mean for NAEP’s future, check out this story by my colleague Sean Cavanagh.)
The report from the Brown Center on Education Policy at Brookings notes that the discrepancy between NAEP and the type of testing expected for the common standards has to do with the varying definitions of an 8th grade math test (or whichever exams for particular subjects and grade levels might be compared). NAEP assesses all of the math that young people have learned through 8th grade; that is, a lot of the content comes from material presumably learned in earlier grades. By contrast, the tests developed to match the common standards are expected to gauge the knowledge and skills learned specifically by the end of each grade level. (Of course, another difference is that NAEP tests only a sample of students, while the common assessments will test virtually all students.)
The report suggests that one way to help minimize the two tests sending contradictory signals about student performance would be to ratchet up the difficulty of NAEP items, bringing the test in closer harmony with the common standards. In the interview, Loveless acknowledged that some NAEP backers may not like this idea, fearing it would disrupt the longitudinal value of NAEP in assessing student achievement over time. However, he said it’s not unprecedented, noting that the National Assessment Governing Board essentially did that with the 12th grade NAEP in math for 2005. “With the 12th grade test, they actually broke the trend [line],” he said.
Another option would be to use adaptive testing to help bridge the gap between the two tests, he said.
The report cautions that there are still a lot of questions as the common core evolves into a set of assessments. (Actually, presumably two sets of assessments, given that two separate consortia of states are currently working to develop new exams.)
“Much work remains to bring the common-core standards to life in a real assessment,” the report says. “Once that happens, an education process will be needed that informs the public and political leaders on what the NAEP and the common core measure, what they have in common, and what differentiates their results.”
In the end, all of this suggests NAEP to be at something of a turning point, the report says.
“A new era is dawning for NAEP. The program has supplied the nation with progress reports on student learning since 1969,” it says. “Now, common-core assessments are on the way. Whether the new assessments push NAEP aside, succeed in augmenting the information provided by NAEP, or force a redefinition of NAEP’s role in monitoring student learning will be at the top of the NAEP policy agenda in the years ahead.”
In the interview, Loveless elaborated: “NAEP was a huge step forward in 1969, and has had to constantly change and modify and reinvent itself along the way to maintain relevance. ... It’s going to have to do that again. The rise of the common-core stands is going to force NAEP into additional changes.”