Assessment

Study Finds NAEP Math Questions ‘Extraordinarily Easy’

By Sean Cavanagh — November 23, 2004 4 min read
  • Save to favorites
  • Print

The level of skill required to solve many of the math questions posed to 4th and 8th graders on the nation’s benchmark of academic progress is “extraordinarily easy,” asserts an analysis unveiled last week.

The report, “How Well Are American Students Learning?”, is available online from the Brown Center on Education Policy. ()

Conducted by the Brown Center on Education Policy, the study also concludes that middle school teachers of mathematics lack academic preparation in the subject and have too few professional-development options available that could help them improve their capabilities.

The report, “How Well Are American Students Learning?,” examines more than 500 questions from the math section of the National Assessment of Educational Progress.

A central finding is that items on the 4th and 8th grade NAEP—a test states must take part in to receive certain federal funds—do not adequately gauge students’ arithmetic ability that is needed to conduct problem-solving on the assessment. The exam puts too much emphasis on testing students’ adeptness to work with whole numbers, the study says, at the expense of fractions, decimals, and percentages that would reflect broader capabilities.

National test scores on NAEP rose in mathematics for both grade levels in 2003, as they have over the past decade. But the study by the Brown Center, a research institute located within the Brookings Institution, a Washington think tank, questions the significance of those gains.

“Even if American students are making progress in mathematics, the arithmetic on which the gains are being registered is not very challenging,” the report contends. “Students know more math than in 1990, but the increase is probably far short of the two years’ worth of knowledge suggested by the NAEP scales.”

Whole Numbers vs. Fractions

The study’s author, Brown Center Director Tom Loveless, focused on 512 NAEP math items that have been made public. He then analyzed the level of arithmetic required to answer correctly questions that test problem-solving skills. Mr. Loveless said he focused on arithmetic because of its importance as both a springboard for learning more complex math and its relevance to real-world problem-solving. Arithmetic is broadly defined as computational skills.

On the 4th grade NAEP, six of 39 questions Mr. Loveless examined required knowledge of 1st-grade-level arithmetic. Overall, those 4th grade questions tested students’ knowledge of arithmetic at a level only slightly above 3rd grade, the analysis found. Yet only 36 percent of the students correctly answered questions at the average level of difficulty, according to the study.

See Also

The 8th grade NAEP questions tested students’ arithmetic skills, on average, at only slightly higher than the 3rd grade level, the study found. Almost 40 percent of the questions tested arithmetic skills taught in 1st or 2nd grade, it concludes.

“We dress up these items as problem-solving,” Mr. Loveless said. “But the fact is that to solve these items, you don’t have to know arithmetic beyond a 3rd grade level.”

But Sharif M. Shakrani, the deputy executive director of the National Assessment Governing Board, which sets policy for NAEP, said the study defines the skills needed in problem-solving in a restrictive way.

Students needed to “identify the appropriate math to use, then use it correctly,” in test questions, he said. “Mr. Loveless does not think [the test measures] arithmetic skills because it measures something in addition to arithmetic skills.”

The Brown Center report recommends that NAEP break out students’ arithmetic skills in its reporting, and that the test-makers reduce the proportion of whole-number items and increase the proportion of fractions, decimals, and percentages. Mr. Shakrani said officials, however, should be cautious about overemphasizing arithmetic at the expense of such subjects as geometry.

Middle School Deficiencies

The Brown Center study also examined the academic skills of middle school mathematics teachers, based on a survey of 252 teachers nationwide.

Of those polled, only 22 percent said they had majored in math in college, and just 41 percent said they had teaching certificates in the subject. The survey has a 6-percentage-point margin of error.

When it came to teacher efforts to improve their skills, only 32 percent of those polled had taken more than five hours of training in algebra, and 17 percent had undertaken that level of work in geometry. More teachers reported seeking some combination of professional development in a variety of classroom-teaching strategies.

The report describes a “lack of focus” in professional-development activities for teachers, with insufficient emphasis on content. Having teachers seek help in two or three well-defined content areas, it says, “would focus resources most effectively.” At least 80 percent of teachers surveyed said that salary increases or stipends would be effective in persuading them to seek such training.

James M. Rubillo, the executive director of the National Council of Teachers of Mathematics, in Reston, Va., said that middle school teachers were more likely than teachers at other grade levels to be bounced among subjects, including mathematics, producing a need for more training. At the same time, he said, providing students with effective middle-grades math instruction is crucial.

“In middle school, the teacher has to build algebraic and geometric thinking,” Mr. Rubillo said. “What [the report] is saying is that people can’t teach what they don’t know.”

Related Tags:

A version of this article appeared in the November 24, 2004 edition of Education Week as Study Finds NAEP Math Questions ‘Extraordinarily Easy’

Events

Jobs October 2021 Virtual Career Fair for Teachers and K-12 Staff
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Data Webinar
Using Integrated Analytics To Uncover Student Needs
Overwhelmed by data? Learn how an integrated approach to data analytics can help.

Content provided by Instructure
Professional Development Online Summit What's Next for Professional Development: An Overview for Principals
Join fellow educators and administrators in this discussion on professional development for principals and administrators.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment State Test Results Are In. Are They Useless?
While states, districts, and schools pore over data from spring 2021 tests, experts urge caution over how to interpret and use the results.
9 min read
FILE - In this Jan. 17, 2016 file photo, a sign is seen at the entrance to a hall for a college test preparation class in Bethesda, Md. The $380 million test coaching industry is facing competition from free or low-cost alternatives in what their founders hope will make the process of applying to college more equitable. Such innovations are also raising questions about the relevance and the fairness of relying on standardized tests in admissions process.
A sign is posted at the entrance to a hall for a test-preparation class. Assessment experts say educators should use data from spring 2021 tests with caution.
Alex Brandon/AP
Assessment Data Young Adolescents' Scores Trended to Historic Lows on National Tests. And That's Before COVID Hit
The past decade saw unprecedented declines in the National Assessment of Educational Progress's longitudinal study.
3 min read
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Assessment Whitepaper
Proven Techniques for Assessing Students with Technology
Dr. Doug Fisher’s proven assessment techniques help your students become active learners and increase their chances for higher learning g...
Content provided by Achieve3000
Assessment Long a Testing Bastion, Florida Plans to End 'Outdated' Year-End Exams
Florida Gov. Ron DeSantis said the state will shift to "progress monitoring" starting in the 2022-23 school year.
5 min read
Florida Governor Ron DeSantis speaks at the opening of a monoclonal antibody site in Pembroke Pines, Fla., on Aug. 18, 2021.
Florida Gov. Ron DeSantis said he believes a new testing regimen is needed to replace the Florida Standards Assessment, which has been given since 2015.
Marta Lavandier/AP