Ky.'s 1998 NAEP Gains Declared Statistically Significant

By David J. Hoff — October 06, 1999 3 min read
  • Save to favorites
  • Print

A second round of research on the latest federal reading-test results suggests that Kentucky’s 1998 scores represented a statistically significant increase over 1994 after all, the U.S. Department of Education announced last week.

By comparing Kentucky 4th graders excluded from the 1998 national assessment with similar students who had taken the state’s own reading test, a department-subsidized researcher concluded that Kentucky’s gains were as large as originally reported, according to a report from the National Center for Education Statistics.

The impact of excluding students with disabilities from last year’s National Assessment of Educational Progress reading test was “modest,” writes Lauress L. Wise, the president of the Human Resources Research Organization, an Alexandria, Va., nonprofit group that conducts education and training research.

Even in the worst-case scenario, the rise in Kentucky’s scores from 1994 to 1998 was “clearly significant,” Mr. Wise writes in the research commissioned by the NCES, a branch of the Education Department.

When the NCES released state-by-state NAEP scores earlier this year, it said Kentucky’s 4th grade scores rose a statistically significant amount. After critics pointed out that the state had excluded higher proportions of disabled students in 1998 than in 1994, the NCES hired researchers to probe whether higher exclusion rates in Kentucky and elsewhere had tainted the states’ assessment results.

An initial review by the Educational Testing Service, which runs the assessment under contract with the Education Department, speculated that scores in Kentucky and Maryland may have benefited from higher exclusion rates. (“Board Won’t Revise State NAEP Scores,” May 19, 1999.)

In response to a request from Kentucky officials, the NCES hired Mr. Wise, who conducts research for the state, to mine data from the former Kentucky testing program to see if he could estimate what would have been the NAEP scores of the 4th graders who were held out of the 1998 exam.

The NCES chose to review Kentucky’s scores because Mr. Wise had access to individual student scores, said Peggy G. Carr, NCES’ associate commissioner in charge of assessment.

While the research did not produce “an exact one-for-one match,” it comes as close as possible to assigning NAEP scores to students who didn’t take the national reading tests, Mr. Wise said.

Apples and Oranges?

But the study paints too rosy a picture of disabled students’ scores, according to the critic who first questioned Kentucky’s gains.

“The basic, fundamental premise is absolutely out to lunch,” charged Richard G. Innes, an airline pilot and a persistent critic of the state’s education policies. “What [Mr. Wise] did is purely and simply compare apples to oranges.”

Mr. Innes estimates that three-fourths of the students with disabilities who were excluded from the national test received such accommodations as having questions read to them when they took the state test. Such extra help, which is not allowed on NAEP, skewed the state results upward, he argues.

“It’s a great way for states to jimmy their NAEP scores,” said Mr. Innes, who lives in Villa Hills, Ky., a suburb of Cincinnati.

Mr. Wise said he didn’t question the results of Kentucky’s test, but tried only to understand how they might have translated to NAEP’s scales.

Gary W. Phillips, the acting commissioner of the NCES, said. the federal agency would not underwrite any other research on the impact of excluding disabled students on last year’s NAEP results.

Instead, the NCES will examine what policy changes it might make in reporting scores, according to Ms. Carr, who oversees the NAEP program.

Commenting has been disabled on effective Sept. 8. Please visit our FAQ section for more details. To get in touch with us visit our contact page, follow us on social media, or submit a Letter to the Editor.


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Teaching Webinar
6 Key Trends in Teaching and Learning
As we enter the third school year affected by the pandemic—and a return to the classroom for many—we come better prepared, but questions remain. How will the last year impact teaching and learning this school
Content provided by Instructure
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Equity & Diversity Webinar
Leadership for Racial Equity in Schools and Beyond
While the COVID-19 pandemic continues to reveal systemic racial disparities in educational opportunity, there are revelations to which we can and must respond. Through conscientious efforts, using an intentional focus on race, school leaders can
Content provided by Corwin
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Equity & Diversity Webinar
Evaluating Equity to Drive District-Wide Action this School Year
Educational leaders are charged with ensuring all students receive equitable access to a high-quality education. Yet equity is more than an action. It is a lens through which we continuously review instructional practices and student
Content provided by BetterLesson

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment Spotlight Spotlight on Assessment in 2021
In this Spotlight, review newest assessment scores, see how districts will catch up with their supports for disabled students, plus more.
Assessment 'Nation's Report Card' Has a New Reading Framework, After a Drawn-Out Battle Over Equity
The new framework for the National Assessment of Educational Progress will guide development of the 2026 reading test.
10 min read
results 925693186 02
Assessment Opinion Q&A Collections: Assessment
Scores of educators share commentaries on the use of assessments in schools.
5 min read
Images shows colorful speech bubbles that say "Q," "&," and "A."
Assessment Standardized Tests Could Be in Jeopardy in Wake of Biden Decisions, Experts Say
Has the Biden administration shored up statewide tests this year only to risk undermining long-term public backing for them?
6 min read
Image of a test sheet.