College & Workforce Readiness

Can NAEP Predict College Readiness?

By Sarah D. Sparks — September 11, 2012 5 min read
  • Save to favorites
  • Print

If you want to know which states are closing black-white achievement gaps in grades 4, 8, and 12, the National Assessment of Educational Progress can show you. If you want to find out how many 8th graders understand how to translate decimals to fractions, “the nation’s report card” can help with that, too.

But after nearly a decade of effort, educators and policymakers are still trying to figure out whether NAEP can predict how likely a state’s students are to start college without needing to take remedial courses, not to mention whether they are prepared for careers. And researchers’ struggles with the federally administered NAEP may highlight the uphill battle that awaits the developers of common state assessments or anyone else trying to tie school performance to the post-high-school world.

“There have definitely been widespread comments over the last few years that there’s no distinction between college and career readiness,” said Louis M. Fabrizio, the North Carolina education department’s director of data, research, and federal policy and a member of the National Assessment Governing Board, which sets policy for NAEP. “What our preliminary research findings indicate is that may not be the case.”

The governing board established a commission in 2002 to decide whether it would be possible to use the 12th grade reading and mathematics NAEP to predict students’ academic preparedness for college, careers, and the military. The final report on the research to date was due out this month, but board members voted this summer to delay its release and include a new round of studies based on the 2013 NAEP.

Cornelia S. Orr, the governing board’s executive director, said the delay is needed because the first five years of the project were filled with “policy discussions of ‘should we, should we not,’ ‘can we, can we not.’ ”

Workplace Relevance of NAEP

BRIC ARCHIVE

To assess the workplace relevance of the National Assessment of Educational Progress, researchers recruited professional trainers from five different career fields to review mathematics test items. In the computer-support field, which generally includes the technicians who repair and service computers, the training specialists found that some of the math topics covered by the test were more useful than others for their line of work.

BRIC ARCHIVE

SOURCE: National Assessment Governing Board

Partly, that’s because the Education Department has already taken heat for using NAEP, which uses samples of students taking portions of the test, to benchmark results from state accountability tests, and the governing board has been gun-shy about setting a particular cut score as definitive.

But there just doesn’t seem to be much definitive when it comes to identifying college and career readiness.

“Our research is really intended to inform the larger national dialogue about how we can assess whether students are prepared, and how that is changing over time,” Ms. Orr said. “The goal for the governing board was to find the point at which students would be prepared academically. As of yet, we haven’t gotten there.”

College Indicators

For college, at least, there are signs NAEP performance may be linked to how well a student will do in initial coursework. Researchers from WestEd, a San Francisco-based research group working under contract to the governing board, found that the 12th grade reading and math tests cover content very similar to that of the SAT.

Moreover, a 2009 study of more than 15,000 12th graders who took both the national assessment and the SAT showed that performing at the proficient level on the math NAEP was associated with an 80 percent chance of earning 500 points out of a possible 800 on the math portion of the SAT, and that the proficient level in reading was associated with a 50-50 chance of scoring 500 on the SAT verbal test.

The SAT has internally pegged a score of 500 to earning at least a B-minus in freshman-level college courses.

NAEP 12th grade content less closely mirrored that used in the ACT, the nation’s other major college-entrance exam; in particular, some arithmetic and applied-math items on the ACT would be covered in more depth on the 8th grade than the 12th grade NAEP in math. NAEP has not been able to compare its performance levels to those in the ACT, though Ms. Orr said the board plans to do so during the 2013 studies, which will also include more state-specific analyses.

Individual states’ data are likely to be critical, North Carolina’s Mr. Fabrizio said, because course requirements vary widely from state to state and even between college systems within the same state.

Hazy Work Picture

The connection between NAEP and preparation for careers that don’t require a four-year college degree is much more tenuous.

The governing board found less overlap between NAEP 12th grade content and that covered on the career-related WorkKeys test, also by ACT Inc. Last spring, panels of professional trainers in five careers—computer-support specialists, automotive master technicians, licensed practical nurses, pharmacy staff, and heating, ventilation, and air conditioning technicians—could not agree on what proficiency level on NAEP would indicate a student was ready for his or her field.

They did agree, however, that most of the content on the test wouldn’t say much about students’ potential in those fields.

For example, “there are hardly any test items in the pool at 12th grade that are applied, based on some use of mathematics rather than theoretical stuff,” said Jeremy Kilpatrick, a co-author of the study and a mathematics education professor at the University of Georgia in Athens. “Where there were such items, the career and technical people were really happy to see that—but most times they looked at the questions and said, ‘This is not relevant to what we want.’”

The assessment governing board will try to bring more clarity around job skills next year, with an analysis that compares the skills and knowledge covered in job-training programs in the five career areas with the math and reading content in the 12th grade NAEP tests.

Still, Ms. Orr was less hopeful about whether NAEP will be useful for gauging career readiness.

The job-training market is very complex, she said. “You find a little more standardization in the auto-mechanic training, but even there it might differ for Dodge or Ford or Hyundai.”

Sean P. “Jack” Buckley, the commissioner of the National Center for Education Statistics, which administers NAEP, said he has not been surprised.

“With [the governing board], it’s a classic problem of a toolbox with only a hammer in it,” Mr. Buckley said. “They’ve got NAEP, and they bang on every policy problem that the world is interested in with their one tool.”

“The right way to do this,” Mr. Buckley said, “is to first develop a framework, say this is college and career readiness, this is what it means, these are the dimensions we think should be included.”

NAEP may not have been designed for college readiness, but Mr. Fabrizio argued it is “the only national indicator we will have for 4th, 8th, 12th grades.”

“I’m not sure that our ability to measure college and career readiness off the 12th grade NAEP is going to make or break anything,” he said. “It will only be complementary to whatever else exists.”

A version of this article appeared in the September 12, 2012 edition of Education Week as Pegging Readiness Remains Elusive Goal for NAEP

Events

Ed-Tech Policy Webinar Artificial Intelligence in Practice: Building a Roadmap for AI Use in Schools
AI in education: game-changer or classroom chaos? Join our webinar & learn how to navigate this evolving tech responsibly.
Education Webinar Developing and Executing Impactful Research Campaigns to Fuel Your Ed Marketing Strategy 
Develop impactful research campaigns to fuel your marketing. Join the EdWeek Research Center for a webinar with actionable take-aways for companies who sell to K-12 districts.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Privacy & Security Webinar
Navigating Cybersecurity: Securing District Documents and Data
Learn how K-12 districts are addressing the challenges of maintaining a secure tech environment, managing documents and data, automating critical processes, and doing it all with limited resources.
Content provided by Softdocs

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

College & Workforce Readiness This East Coast District Brought a Hollywood-Quality Experience to Its Students
A unique collaboration between a Virginia school district and two television actors allows students to gain real-life filmmaking experience.
6 min read
Bethel High School films a production of Fear the Fog at Fort Monroe on June 21, 2023.
Students from Bethel High School in Hampton, Va., film "Fear the Fog"<i> </i>at Virginia's Fort Monroe on June 21, 2023. Students wrote, directed, produced, and starred in the film through a partnership between their district, Hampton City Schools, and two television actors that's designed to give them applied, entertainment industry experience.
Courtesy of Hampton City Schools
College & Workforce Readiness A FAFSA Calculation Error Could Delay College Aid Applications—Again
It's the latest blunder to upend the "Better FAFSA," as it was branded by the Education Department.
2 min read
Jesus Noyola, a sophomore attending Rensselaer Polytechnic Institute, poses for a portrait in the Folsom Library on Feb. 13, 2024, in Troy, N.Y. A later-than-expected rollout of a revised Free Application for Federal Student Aid, or FASFA, that schools use to compute financial aid, is resulting in students and their parents putting off college decisions. Noyola said he hasn’t been able to submit his FAFSA because of an error in the parent portion of the application. “It’s disappointing and so stressful since all these issues are taking forever to be resolved,” said Noyola, who receives grants and work-study to fund his education.
Jesus Noyola, a sophomore at Rensselaer Polytechnic Institute, stands in the university's library on Feb. 13, 2024, in Troy, N.Y. He's one of thousands of existing and incoming college students affected by a problem-plagued rollout of the revised Free Application for Federal Student Aid, or FASFA, that schools use to compute financial aid. A series of delays and errors is resulting in students and their parents putting off college decisions.
Hans Pennink/AP
College & Workforce Readiness How Well Are Schools Preparing Students? Advanced Academics and World Languages, in 4 Charts
New federal data show big gaps in students' access to the challenging coursework and foreign languages they need for college.
2 min read
Conceptual illustration of people and voice bubbles.
Getty
College & Workforce Readiness Learning Loss May Cost Students Billions in Future Earnings. How Districts Are Responding
The board that annually administers NAEP warns that recent research paints a "dire" picture of the future for America's children.
6 min read
Illustration concept of hands holding binoculars and looking through to see a graph and arrow with money in background.
Liz Yap/Education Week and iStock/Getty