Assessment

NAEP Shows Promise as ‘Preparedness’ Yardstick

Research Finds Overlap of 12th Grade Content With That of Other Exams
By Catherine Gewertz — November 22, 2010 3 min read

Includes updates and/or revisions.

Initial studies have delivered early but promising indications that it might be possible to use the exam known as “the nation’s report card” for a brand-new purpose: to gauge students’ preparedness for college or work.

At its quarterly meeting here last week, the National Assessment Governing Board, which sets policy for the National Assessment of Educational Progress, released results of studies comparing the content covered in the 12th grade assessment with the content in the SAT and ACT college-entrance exams; in the Accuplacer, a course-placement test used by colleges; and in WorkKeys, a job-skills test used by employers.

The studies found some differences in the tests’ content, but they also found “considerable overlap.” The overlap is enough to make researchers optimistic, NAGB officials said, about proceeding with the rest of the work needed to make a full determination of whether it would be appropriate to say that certain ranges of NAEP scores correlate with preparedness for work or higher education.

They cautioned, however, that the content analyses alone do not provide enough information to enable that. A flock of related studies is under way to help the board determine whether NAEP can be used to make meaningful statements about career or college preparedness, with a decision slated for late 2011.

Lengthy Inquiry

Among the areas being probed are the statistical relationships between NAEP scores and other exams; the skills measured by NAEP that represent those necessary for entry into certain job-training programs; and how well students at various points on the NAEP scale fare in college and work.

One of the studies contemplated, a comparison of how freshmen at Texas colleges performed on NAEP, will probably be abandoned because too few of those students were willing to sit for the exam in a pilot administration, NAGB officials said. That study could have informed the overall research by, among other things, enabling a comparative look at the scores of students who are placed in credit-bearing courses and those who are placed in remedial courses.

The governing board’s inquiry into the use of NAEP as a college- or work-preparedness indicator grew out of work that began in 2002, when it established a special commission to examine strands of questions about the 12th grade NAEP. As the commission’s work progressed, the question of using the exam as a gauge for endeavors beyond high school took shape. The board approved that program of research in March 2009, right after the 12th grade NAEP on which the research is to be based was administered. (“Plans Advance to Link NAEP to College, Work Readiness”, Dec. 3, 2008.)

Early Results

The “content alignment” studies released Nov. 19 found that the content covered by the 12th grade NAEP is similar to but broader than the content in the SAT, ACT, and Accuplacer exams. Some differences were found in the amount of cognitive rigor the tests demanded of students, the types of reading texts they used, how much each exam used multiple-choice or open-ended questions.

The studies found that “some” of the content of NAEP is similar to that of WorkKeys, but WorkKeys tests some content absent from NAEP. The mathematics and reading skills assessed by WorkKeys, for example, focus on applying skills to workplace settings.

The assessment board’s work flows into an increasingly active dialogue on how to measure college and career readiness. In trying to create its own statements on the issue, the board is taking pains to clarify what its definitions do—and do not—mean.

In a presentation to the board last week, Executive Director Cornelia Orr explained that the term “preparedness” was deliberately chosen to connote the academic skills necessary to qualify for placement into entry-level college courses or job-training programs without remediation. That definition excludes certain skill sets that some others include as part of the “readiness” picture, ranging from work-study habits and teamwork to personal qualities such as persistence.

That approach also differs from some others, in that it focuses on what is necessary to enter college coursework or job-training, not what is necessary to achieve a specific level of success in them. The ACT’s college-readiness benchmarks, for instance, identify threshold scores on the ACT exam that establish the likelihood that students will earn passing grades or better in entry-level, credit-bearing college courses.

Related Tags:

A version of this article appeared in the December 01, 2010 edition of Education Week as NAEP Shows Promise as College-, Career-Preparedness Gauge

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Equity & Diversity Webinar
Culturally Relevant Pedagogy to Advance Educational Equity
Schools are welcoming students back into buildings for full-time in-person instruction in a few short weeks and now is the perfect time to take a hard look at both our practices and systems to build
Content provided by PowerMyLearning
Classroom Technology Webinar Making Big Technology Decisions: Advice for District Leaders, Principals, and Teachers
Educators at all levels make decisions that can have a huge impact on students. That’s especially true when it comes to the use of technology, which was activated like never before to help students learn
Professional Development Webinar Expand Digital Learning by Expanding Teacher Training
This discussion will examine how things have changed and offer guidance on smart, cost-effective ways to expand digital learning efforts and train teachers to maximize the use of new technologies for learning.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment Standardized Tests Could Be in Jeopardy in Wake of Biden Decisions, Experts Say
Has the Biden administration shored up statewide tests this year only to risk undermining long-term public backing for them?
6 min read
Illustration of students in virus environment facing wave of test sheets.
Collage by Vanessa Solis/Education Week (Images: iStock/DigitalVision Vectors/Getty)
Assessment How Can Teachers Better Understand Students? A New Breed of Assessment Will Try to Help
Researchers will work to create formative assessments that can give teachers a window into students’ emerging identities and strengths
4 min read
In this Nov. 19, 2020, file photo, sixth-grade students listen to instruction in class at Cape Cod Lighthouse Charter School in East Harwich, Mass.
Researchers hope to create new assessments to help teachers gain deeper insights into the identities and strengths of their students, like these 6th graders at Cape Cod Lighthouse Charter School in East Harwich, Mass.
Elise Amendola/AP
Assessment Opinion It's Time We Begin Using Assessments to Look Forward, Instead of Back
Schools do not get much value from high-stakes tests. Many are now allowing schools to use better assessments to guide student learning.
Seth Feldman
5 min read
shutterstock 19525837
Shutterstock
Assessment Opinion Grading Has Always Been an Imperfect Exercise. COVID-19 Made It Worse
It’s hard reducing the complexity of each student’s social, emotional, and academic learning to a letter grade. Maybe we’re doing it wrong.
Lory Walker Peroff
4 min read
A student's grades are unknown
Robert Neubecker for Education Week