Assessment

Testing Officials Again Tackle Accommodations And Exclusions for Special Student Populations

By Sean Cavanagh — July 11, 2008 9 min read
  • Save to favorites
  • Print

Perhaps no topic has as thoroughly vexed officials who oversee the nation’s leading test of academic progress as the wide variation among states and cities in the proportion of students with disabilities and limited English proficiency whom they exclude from taking the exam or provide with special accommodations for it.

The board that sets policy for the National Assessment of Educational Progress has been wrestling with the issue for roughly a decade and has made a number of changes in an attempt to close those gaps.

Yet today, the broad discrepancies in the exclusion and accommodation rates of individual states and cities that take part in the heavily scrutinized exam, known as “the nation’s report card,” continue to spark complaints from those who believe those factors skew the results.

Now, an ad hoc committee of the board is once again looking at exclusions and accommodations in the hope of bringing more consistency to those policies. In addition, federal officials have arranged two ongoing studies of the topic, one to determine whether they can identify a model exclusion rate for states, given their student populations, the other to see whether the official reporting of NAEP scores should be changed to better publicize the effect that exclusions are having on different jurisdictions’ scores.

Just four years ago, members of the National Assessment Governing Board, which sets policy for NAEP, approved a measure they hoped would bring more consistency in exclusions and accommodations. Known as a “decision tree,” the policy was aimed at giving local officials clearer guidance on whether students with disabilities or limited English ability should take the test and under what circumstances.

But exclusion and accommodation rates continue to vary widely today, leading some board members to call for more serious changes.

“I would not keep things as they are now,” said Andrew C. Porter, a governing-board member who is serving on the ad hoc committee. That panel is expected to produce recommendations for the full board by next year. State and city policies currently “differ in ways that are problematic,” he said.

“We know what we want. We want valid state-by-state comparisons” that take exclusions and accommodations into account, Mr. Porter added. “How to get them is a little less clear.”

Participation Rising

Accommodations were first allowed on NAEP in 1996. That change came amid a nationwide push among state and local officials to identify and include more students with disabilities on tests and to provide them with special accommodations, such as extra time or the ability to take a test in a more isolated environment. Changes in federal disabilities law encouraged that movement.

Since then, federal officials have sought to include as many students with disabilities and limited proficiency in English as possible on NAEP, with the goal of providing a more complete picture of academic progress within the student population of participating jurisdictions.

Peggy Carr, the associate commissioner of the National Center for Education Statistics, which administers NAEP, noted that federal officials have succeeded in increasing the percentage of students tested in both categories. In 1992, for instance, only 40 percent of students identified as being in those categories participated in the 4th grade reading portion of the test, while today, more than 70 percent do. This is a significant accomplishment, she said, given the demands of testing those populations.

Exclusions: How Strong an Effect?

Unofficial estimates, known as “full-population estimates,” show how jurisdictions would have fared on the National Assessment of Educational Progress in 4th grade reading from 2005 to 2007 if excluded students were included.

BRIC ARCHIVE

Source: Institute of Education Sciences
Full tables for 4th and 8th grade reading and math are available here.

“In the future, I fully anticipate we’ll continue to improve,” Ms. Carr said.

Today, states must participate in NAEP reading and mathematics every two years to receive federal Title I funds. In addition, 11 big-city school districts voluntarily take part in a separate NAEP Trial Urban District Assessment. Seven additional cities will join that test soon.

Still, recent test results have revealed wide gaps in the percentages of students that states and districts exclude from NAEP, or provide with special accommodations. For instance, on the 2005 urban tests in 4th grade reading, two Texas districts, Austin and Houston, excluded 20 percent and 23 percent of students with disabilities or limited English skill, respectively, compared with just 4 percent in Atlanta and 6 percent for Los Angeles and New York City. Federal officials made special note of the Texas districts’ higher exclusion rates in reporting the data.

Those gaps also have been evident on NAEP state results in reading and math. For example, Massachusetts excluded 9 percent of students in both categories combined, in grade 8 math in 2007, while California excluded only 2 percent of them.

IEP Conflicts

But federal attempts to bring more uniformity to exclusion and accommodation rates are complicated by the mishmash of state and local policies governing the issue.

One major obstacle, for instance, is that there are no uniform policies across states or cities on which students are classified as having a disability or lacking English-language skill. As a result, the percentage of students identified as belonging to either category on NAEP varies across jurisdictions.

Another complicating factor: Decisions about what special testing accommodations students with disabilities receive are left to local education officials, who rely heavily on students’ individualized education programs, or IEPs. Those plans, mandated by the federal government for students who receive special education services, are written by school officials and parents.

Sometimes, a student’s IEP calls for a testing accommodation that is allowed on state tests but not on NAEP. One of the most common examples of this disconnect occurs in reading, federal officials say. Some states allow students with certain disabilities to have passages on state reading assessments read aloud to them. But federal officials do not permit that accommodation on NAEP, reasoning that such a practice amounts to a test of students’ listening skills, rather than their reading comprehension.

In those cases, local officials may simply choose to exclude those students from taking part in NAEP. Federal officials have encouraged local officials to consider allowing those students to take part in NAEP anyway, even without the accommodations specified in their IEPs. They also note that individual students’ test scores are not reported on NAEP, as they are on individual state assessments.

Yet local officials are often reluctant to stray from the recommendations of the IEP and permit students to take part in NAEP. That wariness was evident in a 2007 federal study, which found that local decisionmakers typically believe that administering NAEP without accommodations would “constitute a violation of the student’s rights.”

Full-Population Estimates

In an attempt to bring more consistency to exclusion and accommodation rates, the National Assessment Governing Board in 2004 approved the use of a “decision tree” designed to help state and local officials determine which students should participate on NAEP with or without accommodations.

The decision tree asked local officials to consider a series of questions, such as whether a student with disabilities took his or her state’s assessment with or without accommodations, or took a modified test, in making a final decision about NAEP participation. But that process is still contingent on dissimilar state and local testing policies.

“It’s not solved the problem,” said Mr. Porter, the governing-board member, who is also dean of the graduate school of education at the University of Pennsylvania. “We wouldn’t be dealing with this if it had stopped the problem.”

One option would be to have federal officials produce and report statistical estimates for how states’ and cities’ NAEP scores would change if all their students—including those excluded—had participated, or hadn’t been accommodated.

Federal officials already publish a version of those calculations for excluded students, known as “full-population estimates,” on the NAEP Web site. For most states and cities, the impact of incorporating their excluded students in test results would have been relatively small, those estimates show. For instance, in 4th grade reading in 2007, the scores of only three states, plus the District of Columbia, would have risen or fallen by 2 or more points, on a 500-point NAEP scale, from the previous testing year, on the basis of estimates of how their excluded students would have fared.

But the impact of exclusions would have altered scores in some jurisdictions. The District of Columbia’s 2007 gains in 4th grade reading on the state NAEP would have gone from a 6.3-point increase, which was statistically significant, to a 3.1-point increase, which was not. Louisiana’s reading score, on the other hand, would have swung from a 1.8-point decrease to a 2-point gain, though neither change was statistically relevant.

The full-population estimates were developed for federal officials by Donald McLaughlin, formerly a chief scientist at the American Institutes for Research, in Washington, who is now retired. Mr. McLaughlin argues that the full-population estimates, which are based on assumptions about how excluded students would have performed, should be presented as the official NAEP scores, rather than simply shown in online tables.

While the effect of state and city exclusion rates varies with each administration of NAEP, those differences undermines the assessment’s credibility, he argued.

“The impact varies from state to state, grade to grade, subject to subject, and year to year,” he said in an e-mail. “The only way to get rid of the problem, and not to have the uncertainty of exclusion hanging over NAEP’s head, is to use the full-population estimates.”

Studies Under Way

But while Ms. Carr of the NCES said she believes online estimates are statistically sound, she also notes that they are only “hypotheticals about what students might have done,” rather than the scores of actual students who took NAEP. Therefore, federal officials should be cautious before presenting them as official results. To accept the scores, she said, “you have to be able to swallow the premise” of the estimates.

Federal officials have commissioned a study to determine whether it makes sense to include those estimates in the official NAEP reports. That work is being conducted by an independent research organization, the National Institute of Statistical Sciences, in North Carolina.

The second, ongoing federal study is attempting to determine whether it is possible to set reasonable NAEP exclusion rates, for individual states, given their accommodation policies and populations of students with disabilities, Ms. Carr said. A similar study could be performed later for urban districts, NCES officials say.

Mr. Porter and some other governing-board members would like to see federal officials attempt to establish a uniform exclusion and accommodation policies for NAEP across jurisdictions. But others question whether that is feasible and whether states would agree to such measures.

One way of getting around such reluctance would be to have states agree to voluntary exclusion and accommodation policies, said board member Alan J. Friedman. “I’d much rather see the states come together themselves,” said Mr. Friedman, a New York-based consultant on science education issues. “It would be better than it is now.”

A version of this article appeared in the July 16, 2008 edition of Education Week as Testing Officials Again Tackle Accommodations And Exclusions for Special Student Populations


Commenting has been disabled on edweek.org effective Sept. 8. Please visit our FAQ section for more details. To get in touch with us visit our contact page, follow us on social media, or submit a Letter to the Editor.


Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Teaching Webinar
6 Key Trends in Teaching and Learning
As we enter the third school year affected by the pandemic—and a return to the classroom for many—we come better prepared, but questions remain. How will the last year impact teaching and learning this school
Content provided by Instructure
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Curriculum Webinar
How Data and Digital Curriculum Can Drive Personalized Instruction
As we return from an abnormal year, it’s an educator’s top priority to make sure the lessons learned under adversity positively impact students during the new school year. Digital curriculum has emerged from the pandemic
Content provided by Kiddom
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Equity & Diversity Webinar
Leadership for Racial Equity in Schools and Beyond
While the COVID-19 pandemic continues to reveal systemic racial disparities in educational opportunity, there are revelations to which we can and must respond. Through conscientious efforts, using an intentional focus on race, school leaders can
Content provided by Corwin

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Assessment Whitepaper
Proven Techniques for Assessing Students with Technology
Dr. Doug Fisher’s proven assessment techniques help your students become active learners and increase their chances for higher learning g...
Content provided by Achieve3000
Assessment Long a Testing Bastion, Florida Plans to End 'Outdated' Year-End Exams
Florida Gov. Ron DeSantis said the state will shift to "progress monitoring" starting in the 2022-23 school year.
5 min read
Florida Governor Ron DeSantis speaks at the opening of a monoclonal antibody site in Pembroke Pines, Fla., on Aug. 18, 2021.
Florida Gov. Ron DeSantis said he believes a new testing regimen is needed to replace the Florida Standards Assessment, which has been given since 2015.
Marta Lavandier/AP
Assessment Spotlight Spotlight on Assessment in 2021
In this Spotlight, review newest assessment scores, see how districts will catch up with their supports for disabled students, plus more.
Assessment 'Nation's Report Card' Has a New Reading Framework, After a Drawn-Out Battle Over Equity
The new framework for the National Assessment of Educational Progress will guide development of the 2026 reading test.
10 min read
results 925693186 02
iStock/Getty