Assessment

NAEP Data on Vocabulary Achievement Show Same Gaps

December 06, 2012 5 min read

Includes updates and/or revisions.

A new analysis of federal data that provide a deeper and more systematic look into students’ ability to understand the meaning of words in context than was previously available from “the nation’s report card” finds stark achievement gaps in vocabulary across racial and ethnic groups, as well as income levels. The analysis aims to offer greater insights into reading comprehension.

The first-of-its-kind National Assessment of Educational Progress report suggests a consistent relationship between performance on vocabulary questions and the ability of students to comprehend a text, which experts say is consistent with prior research on the subject.

In 2011, 4th and 8th graders performing above the 75th percentile in reading comprehension on NAEP had the highest average vocabulary scores, the report says. Likewise, those 4th and 8th graders scoring at or below the 25th percentile had the lowest average vocabulary scores.

“Today’s special report puts an important spotlight on something that’s not discussed nearly enough on its own: vocabulary,” Brent Houston, the principal of Shawnee Middle School in Shawnee, Okla., and a member of the NAEP governing board, said in a statement last week. “We discuss concepts such as reading comprehension and reading on grade level, but we can’t have success in those areas if our students also do not learn to understand the meaning of words in a variety of contexts.”

What was especially troubling, Mr. Houston said, were the achievement gaps identified in the report.

“Perhaps what struck me most—and what hits closest to home—is observing the performance trends by family income,” he said.

As Mr. Houston pointed out, the data reveal large gaps in vocabulary achievement between students who are eligible for a free or reduced-price lunch and those who are not. In 4th grade, the gap was 31 points on a 0-500 scale. In 8th grade, the gap was 28 points.

The report does not provide achievement levels for students, such as “proficient” or “basic,” as is typical for NAEP reports. Data from the broader NAEP reading report for 2011 found just 34 percent of both 4th and 8th graders scoring at or above the proficient level.

“Schools nationwide really need to go beyond teaching word definitions” to improve reading performance, Mr. Houston said.

The new report offers a sampling of vocabulary words that tripped up many students. The word “permeated” was a trouble spot for a lot of 8th graders, with nearly half failing to correctly identify its meaning in a nostalgic passage about eating a “mint snowball” at a small-town drugstore. And “puzzled” was apparently puzzling for 49 percent of 4th graders, who misidentified its meaning in a passage from the story “Ducklings Come Home to Boston.”

‘The Early Stages’

A revised NAEP framework for reading, instituted in 2009, seeks to provide a more detailed and “systematic” measure of vocabulary. While previous reading assessments had included some vocabulary questions, the revised framework set new criteria for developing vocabulary questions and increased their number. The changes, a NAEP fact sheet says, allow the test to “reliably measure students’ vocabulary performance and report it separately.”

Vocabulary questions were multiple-choice and appeared in two different sections of the reading exam: comprehension and vocabulary.

Margaret McKeown, a senior scientist for learning research and development at the University of Pittsburgh, said in a statement that the new assessment is distinct from traditional vocabulary exams in three ways. First, it’s not based on a list of specific words. Second, the “target words” appear within the context of a passage, “rather than in isolation.” And third, the NAEP items emphasize an understanding of a word’s use within a given context, rather than the definition of the word on its own.

“This decision represents the major rationale for the assessment,” Ms. McKeown said, to measure “the kind of knowledge that students need to have about words in order to use the words to understand what they read.”

She added: “Although we are in the early stages of assessing vocabulary in NAEP, these initial results may give us some clues on patterns and how vocabulary fits into reading comprehension. ... Future NAEP reports in this area will provide invaluable data and trends on vocabulary in text that provide a better grasp of the nature of comprehending text and the role vocabulary knowledge plays in the quality of comprehension.”

Ms. McKeown served on a NAEP planning committee charged with developing recommendations for the current reading-assessment framework.

The report includes achievement data for 2009 and 2011 at grades 4 and 8. The average overall score did not shift by a statistically significant margin at either grade level. But there were changes in certain categories. For example, the lowest-achieving 8th graders, those at the 10th percentile, saw a gain of 2 points on the NAEP scale, which was statistically significant.

On the issue of achievement gaps by race and ethnicity, the analysis found that in 2011, black students trailed white students, on average, by 29 points in both the 4th and 8th grades. Changes from 2009 to 2011 were not deemed statistically significant.

Meanwhile, Hispanic 4th and 8th graders also trailed their white peers, by 28 points in 8th grade and 29 in 4th grade in 2011.

Girls outperformed boys by slight margins in grades 4 and 8 (2 points and 3 points, respectively) in 2011. The 1-point difference in 12th grade, from the 2009 assessment, was not statistically significant. In 2011, 12th graders were not tested.

‘Barren’ and ‘Eerie’

A chart featured in the report highlights some of the vocabulary words tested and how students fared in recognizing their meaning in context.

In grade 4, words like “barren,” “detected,” and “eerie” posed problems, with fewer than half of students correctly identifying their meaning. But “created,” “spread,” and “underestimate” were correctly understood by 75 percent or more.

The word “urbane” was difficult for both 8th and 12th graders, with fewer than half getting the correct answer. But “anecdotes” was correctly understood by three-quarters of 8th and 12th graders.

Several criteria were used to select words for inclusion in the vocabulary questions, according to the report. Those words were to be: characteristic of written language, as opposed to everyday speech; used across a variety of content areas, rather than being technical or specialized language; generally familiar concepts, feelings, or actions; and necessary for understanding part or all of a passage.

A version of this article appeared in the December 12, 2012 edition of Education Week as Brand-New NAEP Report on Vocabulary Shows Same Old Gaps

Events

Classroom Technology Webinar Making Big Technology Decisions: Advice for District Leaders, Principals, and Teachers
Educators at all levels make decisions that can have a huge impact on students. That’s especially true when it comes to the use of technology, which was activated like never before to help students learn
Professional Development Webinar Expand Digital Learning by Expanding Teacher Training
This discussion will examine how things have changed and offer guidance on smart, cost-effective ways to expand digital learning efforts and train teachers to maximize the use of new technologies for learning.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
The Social-Emotional Impact of the Covid-19 Pandemic on American Schoolchildren
Hear new findings from an analysis of our 300 million student survey responses along with district leaders on new trends in student SEL.
Content provided by Panorama

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment Opinion Q&A Collections: Assessment
Scores of educators share commentaries on the use of assessments in schools.
5 min read
Images shows colorful speech bubbles that say "Q," "&," and "A."
iStock/Getty
Assessment Standardized Tests Could Be in Jeopardy in Wake of Biden Decisions, Experts Say
Has the Biden administration shored up statewide tests this year only to risk undermining long-term public backing for them?
6 min read
Illustration of students in virus environment facing wave of test sheets.
Collage by Vanessa Solis/Education Week (Images: iStock/DigitalVision Vectors/Getty)
Assessment How Can Teachers Better Understand Students? A New Breed of Assessment Will Try to Help
Researchers will work to create formative assessments that can give teachers a window into students’ emerging identities and strengths
4 min read
In this Nov. 19, 2020, file photo, sixth-grade students listen to instruction in class at Cape Cod Lighthouse Charter School in East Harwich, Mass.
Researchers hope to create new assessments to help teachers gain deeper insights into the identities and strengths of their students, like these 6th graders at Cape Cod Lighthouse Charter School in East Harwich, Mass.
Elise Amendola/AP
Assessment Opinion It's Time We Begin Using Assessments to Look Forward, Instead of Back
Schools do not get much value from high-stakes tests. Many are now allowing schools to use better assessments to guide student learning.
Seth Feldman
5 min read
shutterstock 19525837
Shutterstock