The board that oversees the National Assessment of Educational Progress has approved a series of policy changes, some potentially controversial, to help bring the federally financed testing program into line with new requirements under the revised Elementary and Secondary Education Act.
During its action-packed quarterly meeting, held here May 16-18, the National Assessment Governing Board also endorsed the idea of testing a nationally representative sample of charter school students in reading and mathematics in 2003. The board will not make a final decision until August.
|
Chester E. Finn Jr., the president of the Washington-based Thomas B. Fordham Foundation and a former chairman of the governing board, made the request on behalf of a loose coalition of groups that support charter schools, which are self-governing public schools. Although nearly 2,400 such schools now operate in the United States, he noted, enrolling almost 580,000 students, no national data on student achievement in them exist.
“I’d like to see us add the weight of our support to this proposal,” said Michael E. Ward, the state superintendent of public instruction in North Carolina and a member of the board, also known as NAGB. “This is a major public-policy issue, and we have, at least on a national level, an absence of performance data.”
|
But Gary W. Phillips, the acting commissioner of the National Center for Education Statistics, the Department of Education branch that oversees NAEP, cautioned that it might not be possible to report on the performance of such schools in 2003.
“We have to have an accurate list of the population of schools from which we can draw a sample,” he said, noting that, by the time the federal government gets the name and address of a charter school, sometimes it has gone out of business. In addition, he pointed out that the field test for the 2003 NAEP assessment has already been concluded.
The NCES will try to identify a nationally representative sample of charter schools by the August meeting, so that board members can decide whether assessing those schools in 2003 is feasible.
Meanwhile, NAGB voted to cut back on the number of private school students included in the national assessment. Since 2000, the national sample has been augmented with additional numbers in order to yield achievement data on several categories of private schools, beyond those that are Roman Catholic. Although NAEP will continue to oversample private school students in grades 4 and 8, the board voted to stop oversampling 12th graders because school participation rates have been too low to generate reliable results. The decision could save approximately $2.5 million next year.
Representatives of several private school groups who attended the meeting said the board’s action caught them by surprise.
“I’m a little frustrated that, without much notice, the 12th graders are disappearing,” said John C. Holmes, the director of government affairs for the Association of Christian Schools International.
Joe McTighe, the executive director of the Council for American Private Education, said he was disappointed that such an important decision would be made without any consultation with the private school community.
Policy Changes
The debates over who should be included in NAEP, known as the “nation’s report card,” underscore just how important the federally financed testing program has become. The program periodically tests, and surveys, national samples of students on a number of core subjects.
Under the “No Child Left Behind” Act of 2001, NAEP will test a representative sample of 4th and 8th graders in reading and math in each state every other year, with the federal government paying the cost of administering the exams. The results will be used, at least informally, to help gauge whether state standards and tests are rigorous enough.
At its meeting, the board adopted a number of policies to bring the national assessment in line with the new legislation that reauthorizes the ESEA, with further action anticipated in August:
- Long-term trend: The new law requires the government to continue collecting trend data on reading and math achievement dating back to the 1970s. The policy approved by the board this month provides for the continuation of the long-term trend in reading, math, and science.
NAGB also approved a major “bridge” study for 2004 to determine how any changes made in the assessments—such as permitting some students to take the tests with accommodations—might affect the results. Richard G. Innes, an independent researcher based in Kentucky, warned, “They absolutely are going to trash the validity of the long-term trend if they permit accommodations.”
But Sharif M. Shakrani, the deputy executive director of NAGB, said, “If the accommodations will impact the results, we will find out from the bridge study,” which is designed to compare similar populations of students who take the assessment using the old and the new methods of collecting data. Analyses of students who took the main NAEP with and without accommodations in 1996 found no statistically significant difference in test results, he said.
- Public access: The reauthorized ESEA requires that the public have access to all NAEP questions, under secure conditions. The new policy specifies that individuals will have the right to examine NAEP items within 45 days of a written request, at a secure location, as required by law.
Several board members expressed alarm that the requirement could result in breaches of security and prove unwieldy, particularly if large numbers of people ask to review the exams. “I have to say, just for the record, I think this is an idiotic policy,” said Diane Ravitch, a member of the board and a senior research scholar at New York University.
“I think we have to abide by the law,” said Mr. Shakrani of NAGB. “At the same time, we will have to be cautious to ensure that the implementation of the law will not endanger the security of the test.”
- Complaint process: The law also requires the board to establish a formal process for reviewing complaints about NAEP. Under the policy, the board’s executive director will review all written complaints, which may be appealed to the governing board. The law requires that all decisions be submitted to the secretary of education, who will in turn report to Congress.
- Background data: The 2001 law gives the governing board “final authority” to approve all background questions on NAEP, transferring that responsibility from the education statistics center. The ESEA also contains new language that all NAEP questions must be “secular, neutral, and nonideological.” The new NAGB policy specifies that the collection of background data should be limited and directly related to academic achievement, or to the fair and accurate presentation of achievement results. The policy approved by NAGB also spells out what is meant by “secular, neutral, and nonideological,” with specific examples.
- Subject-area frameworks: NAGB also revised its policy for developing the subject-area frameworks that determine the content and format of NAEP tests, as well as its policy for the review and approval of test items. The law no longer requires the governing board to reach a national consensus in the development of test frameworks, although the policy pledges to use a “comprehensive, inclusive, and deliberative” process.
Reorganization Criticized
While all the changes are designed to maintain the integrity and independence of NAEP, board members expressed concern that legislation pending before Congress could do just the opposite.
Under HR 3801, approved by the House earlier this month, the commissioner of education statistics would be appointed by the director of a new “Academy of Education Sciences,” rather than by the president. It also would give the director the authority to award NAEP contracts, grants, and cooperative agreements “acting through the commissioner.” And it would submit all academy reports, including NAEP results, to the secretary of education 30 days before their release. (“Research Bill Clears House Without Fuss,” May 8, 2002.)
“The bill, as it is currently written, reduces the independence of both NAGB and NCES and removes some of the political insulation of both of these agencies,” contended Ms. Ravitch, who served as an assistant secretary of education during the first Bush administration. “These are both agencies that are truth-telling agencies— assessment and statistics—and both should be insulated to the maximum extent possible from any political controls.”
Board members noted that there are several places in the legislation where Congress reiterates its intent to maintain the current relationship between the governing board and the federal department of education.
“We don’t want to nitpick,” said Mark D. Musick, the chairman of NAGB, “but we don’t want to pretend there aren’t issues, if indeed there are issues.”
