Over the objection of officials at the statistical wing of the U.S. Department of Education, the independent body that sets policy for the National Assessment of Educational Progress has approved a policy that significantly narrows the grounds for excluding students with disabilities and English-language learners from the exams.
Beginning as early as next year, the National Assessment Governing Board will also highlight states and districts that don’t meet testing-participation targets for those populations.
Adopted March 6, the new policy caps a decadelong effort to increase participation of students with disabilities and ELLs in the federally sponsored assessment and to bring uniformity to exclusion and accommodations decisions on NAEP for those students.
Until now, those decisions have essentially been directed by a patchwork of state and local policies, resulting in wide variations in exclusion rates. Guidance established by NAGB in 2004 tried to standardize decisionmaking, but still allowed local officials to determine whether, for instance, a student would still take NAEP if the national exam did not permit the same accommodations as state tests.
The disparities have long been worrisome for testing experts, who say they could jeopardize NAEP’s role as a uniform measuring stick of student achievement across states and cities.
The 24 members of the National Assessment Governing Board had an unusually busy agenda during its most recent meeting. In addition to the new policy governing NAEP inclusion and accommodations policies, the board approved:
• An alteration to the schedule of assessments. The board will add a science assessment at grade 8 in 2011 at the state and national level. The addition will allow the National Center for Education Statistics to conduct a “linking study” between the exam results and those on the Trends in International Mathematics and Science Study, better known as TIMSS, to be assessed that year.
• A plan to cancel the administration of the national and state writing assessment in grade 4 for 2011, to free up funding for the science exam. Moreover, the writing assessment was to have been given in a pencil-and-paper format in 2011 and subsequently changed to a computer-based format, which raised concerns about an interruption in the trend line.
• New achievement-level descriptors for the 2009 reading exam, to reflect its updated framework.
• A framework for the naep Assessment in Technology and Engineering Literacy, which will guide the development of the test, to be given at one or more grades in 2014.
SOURCE: Education Week
Though supportive of the goal of greater inclusion on NAEP, the acting commissioner of the National Center for Education Statistics indicated that his agency couldn’t support the policy. In a letter drafted days before the board vote, Stuart Kerachsky said that NCES harbors concerns about “flagging” individual states’ exclusion rates.
“There is no statistical basis for such standards,” he wrote. “For that reason alone, NCES is unable to support this recommendation: We would be implicitly impugning jurisdiction results ... without cause.”
Additionally, he warned that the policy could be viewed as an attempt by the board to sway state instructional decisionmaking, in contravention of federal law.
But those arguments didn’t convince NAGB Chairman David P. Driscoll or the other board members, who approved the policy unanimously.
“We have been at this for almost two years,” said Mr. Driscoll, a former Massachusetts state education chief, referencing the work of the ad hoc committee set up by the board to craft recommendations for the policy. “To do nothing would be totally wrong, and the targets here are very reasonable.”
Fewer Exclusions
The policy narrows the grounds for school officials to disallow student participation on NAEP, sometimes called “the nation’s report card.” The exam uses a methodology involving representative samples of students to arrive at its scores.
Under the new policy, states and districts would aim to test at least 95 percent of all students in the sample, and to test 85 percent of the students with disabilities and ELLs. States that didn’t reach the overall participation goal would be “prominently designated” in NAEP reports, the policy states, while those not meeting the participation rates for special education students or ELLs would be “identfied” in the reports.
For students with disabilities, the guidelines state that only those students who have been identified as having the most significant cognitive disabilities should be considered for exclusion.
All other students would participate, using accommodations specified on their Individualized Education Plan or Section 504 plan. If a specified accommodation is not permitted on NAEP for reasons of test validity—such as read-aloud assistance on the reading exam, or the use of a calculator on math—then the student would be encouraged to take the exam without that accommodation. Students could still refuse to participate.
For ELLs, the guidelines state that students who have been in the United States for a year or more should be included. A student in the country for less than that could take an exam if it is available in his or her native language. School staff members would select among ELL accommodations using “objective indicators” of English proficiency, the guidelines state. The board also directed the NCES to create assessments in Spanish for all subjects other than reading and writing.
Sticky Issues
Several hard-to-pin down issues are at the center of the debate. One concerns uncertainty over just how much exclusion rates affect overall scores on NAEP.
“We know that there is a nonsignificant relationship between exclusion rates and student scores,” said Peggy G. Carr, the associate commissioner for NCES’ assessment division. “We do know, however, that there is a moderate, mostly significant relationship between changes in exclusion rates and changes in scores.”
States are required to take part in NAEP math and reading at both grades 4 and 8.
But according to Mr. Kerachsky, although exclusion rates have fallen, few states have managed to reach the targets specified in the policy. Based on 2009 participation data on reading, math, and science tests, only three states would have met the targets, he pointed out.
In his letter, Mr. Kerachsky stated that the policy could be seen as an effort by the board to influence state and local policies around exclusion. Language in the federal No Child Left Behind Act, he noted, prohibits the federal government from using NAEP to influence states’ “instructional practices.”
Already, state and district exclusion rates are published in tables at the back of the NAEP reports, he noted. “I think we all agree exclsion is an important issue,” Mr. Kerachsky told the board. “It’s a question of the importance of a flag or a designation. You are implying then that there is something wrong with the [test] results.”
Though his arguments made little headway with the board, Mr. Kerachsky’s position has been supported by other stakeholders. A policy task force made up primarily of state education department officials, convened by NAGB and the Council of Chief State School Officers, urged the govern
ing board to report the 95 percent inclusion goal as a target, but to hold off on flagging individual states’ rates for the time being.
A handful of members who supported the new policy urged careful oversight as it is implemented, especially with regard to encouraging students with nonallowable accommodations to test without them.
“ ‘Encouragement’ is an endeavor that can be pursued with varying degrees of yield,” said W. James Popham, a member of the governing board. Some degree of training for local test administrators and oversight to ensure that they don’t go “overboard” with the policy should be instituted, he said.
Yet another fine line involves concerns around the IEP, a legal document that specifies the services and accommodations to which students with disabilities are entitled.
The original draft would have directed some students to take exams without their IEP-specified accommodations, but that language was softened because of legal concerns, said Oscar Troncoso, the chairman of the ad hoc committee.
Mr. Kerachsky’s letter warns that the legal status of the IEP could still be a potential problem, but some attorneys dispute that assertion.
“By allowing only accommodations that maintain NAEP’s validity, regardless of what accommodations a student’s IEP may include, NAGB’s new policy is consistent with well-written IEPs and with legal requirements in special education and the NCLB,” said Miriam K. Freedman of the Boston-based law firm Stoneman, Chandler & Miller llp, who specializes in special education law and served as a member of the task force on students with disabilities.
And several members of the governing board indicated that they were influenced by widespread support for the guidelines among parents and advocacy groups at public hearings.
Those stakeholders “spoke very, very strongly in favor of the guidelines,” said NAGB member Susan Pimentel, an educational consultant. “I think there is a sense and a perception of basic fairness here, and that perception is so important to the credibility of NAEP.”
The policy got an additional boost from Alexa E. Posny, the deputy assistant secretary for the Department of Education’s office of special education and rehabilitation services, who called it an “important step forward.”
“Inclusion does not happen on its own,” Ms. Posny said. “It is hard work.”
Ms. Posny chaired the task force on students with disabilities that helped draft the initial policy. At that time, she was Kansas’ schools chief and had only been nominated for the federal post.
Uncertainty Ahead?
Before being implemented, the proposal must first pass legal muster by the Department of Education’s office of the general counsel.
“We really get the legal issue nailed down, and then we explore whether there’s a way to accomplish NAGB’s goals within the boundaries of our charter,” said John Q. Easton, the director of the Institute of Education Sciences, of which the NCES belongs. “I don’t know what that [way] is, and we won’t know until we get down to trying different approaches.”
He sought to underscore that that work will be done in partnership with NAGB. Nevertheless, the meeting highlighted the delicate—and at times uneasy—relationship between NAGB and the NCES. Ambiguities in how the two bodies are structured make it difficult to determine which of them is supposed to follow the other’s lead.
Of the two, the NCES is generally viewed as more cautious about statistical reporting, to ensure that information is not misinterpreted or accused of being partisan. NAGB, on the other hand, has pushed to make the reports accompanying NAEP releases more user-friendly. It successfully introduced descriptors of achievement at three levels—“basic,” “proficient,” and “advanced”—in NAEP releases, but those descriptors are printed with a disclaimer from the NCES urging caution in interpretation.
“My view is that NCES has to stand behind its reports,” Mr. Kerachsky said, when asked which agency ultimately would get the last say over implementation of the policy. “We take advice from NAGB, but these are government reports that have to meet guidelines from [the Office of Management and Budget], for example, for statistical reporting.”
But in an interview, Mr. Driscoll, made it clear that NAGB expects the NCES’ cooperation in carrying out the policy, including the identification of states whose exclusion rates remain high. “It doesn’t mean we’re going to give states a scarlet letter,” Mr. Driscoll said, “but it certainly doesn’t mean putting these rates in the back of the report where no one will see them.”
