National Academy of Sciences Is Asked To Evaluate Work of Statistics Agency
Washington--The advisory panel responsible for overseeing the work of the National Center for Education Statistics has asked the National Academy of Sciences to conduct the first major evaluation of the agency in its 12-year existence.
The action comes at a time when the center, under criticism for having disseminated vocational-education data of questionable quality, is facing reauthorization and operating with a reduced budget.
Although the contract for the evaluation has not yet been awarded, officials of the academy and of the National Advisory Council on Education Statistics said it would address such topics as the agency's research methodologies and the reliability, timeliness, and usefulness of its data. They said the proposed evaluation of the Education Department's statistics-gathering branch would take from 18 months to two years to complete.
The advisory panel is scheduled to receive a progress report on the proposed study at its meeting here this week.
Central to Federal Role
The evaluation would focus on an activity that President Reagan and federal education officials have repeatedly cited as central to the federal role in education. Some observers believe it could influence the outcome of Congressional proposals to extend the life the education-statistics agency.
Although the House approved a measure in July that would reauthorize the nces through the end of fiscal 1989, the Senate has declined to act on the bill. Another reauthorization bill is likely to be reintroduced after the new Congress convenes in January.
Concerns About Quality
Members of the Congress and users of data produced by the nces and other federal statistical agencies have voiced concerns about the effects of budget reductions in recent years on the quality of those data.
The budget for the nces, for example, has shrunk from about $10 million in fiscal 1980 to $8.75 million in the fiscal year that ended on Sept. 30. According to officials at the agency, those reductions resulted in the loss of about 50 staff positions from a peak level of about 170.
Fiscal 1985 funding bills working their way through the Congress last week would preserve the center's current level of funding.
In a report released in late July, the General Accounting Office noted that although the nces was able to preserve its core survey programs in spite of budget cuts, it "significantly reduced its efforts" to determine whether its studies were methodologically sound.
In 1980, the gao reported, the center spent about $1.5 million, or 15.1 percent of its funds, on statistical methodology. In the following year, the amount spent on such activities was reduced to $239,000, or 2.7 percent of the center's budget.
Problems With veds
Questions about the reliability of nces data were also raised in 1983 when the Office of Management and Budget, citing serious year-to-year inconsistencies, ordered the center to suspend the collection of data under its Vocational Education Data System. Last March, the budget office turned down an appeal by the Education Department that it be allowed to comply with a law passed by the Congress in 1976 mandating the collection of the veds data.
According to Marie D. Eldridge, who resigned as administrator of the nces earlier this year, the prescriptive nature of the law that created veds all but ensured that states would have a difficult time completing the survey accurately.
"I think it is to the center's credit that when the staff first had hard evidence of problems with the data, it made it known in the department that the data were of very questionable quality," Ms. Eldridge said in defense of the center's quality-control procedures.
But according to Robert Barnes, whose work in the Education Department's office of planning, budget, and evaluation required him to use the veds data, the fact that the agency disseminated "admittedly wretched data" without a disclaimer points to a serious quality-control problem.
"veds was the litmus test of the center's dissemination standards, and they certainly didn't pass it," Mr. Barnes said.
"With any statistical agency, questions about quality control always come up," noted Donald J. Senese, the department's assistant secretary for educational research and improvement and the nonvoting chairman of the advisory panel on education statistics.
"There's the question of the veds data," he continued. "Is that a quality-control problem? Some say the problem is in the way that Congress set the system up. Others say there were things that the nces could have done to change it."
"No agency can produce absolutely perfect data. There will always be errors," said Ms. Eldridge, who now heads the Center for Education Studies at the Research Triangle Institute in North Carolina. "I believe that what will come out of this proposed study of nces is that the quality-control procedures that it has established are as rigorous or more rigorous than at comparable agencies. Given current staffing levels and the skill mix, the center will probably receive accolades in this area."
According to Vance Grant, who heads the center's statistical-information office and whose career with the precursor to the nces began in 1955, "on the dissemination side, my general impression is that our quality is quite high."
"When you're doing surveys of all the colleges and unversities in the country, for example, and you get a response rate of almost 100 percent, you don't have a problem," he said.
"People are beating a path to our door for our data," Mr. Grant continued. "This office gets about 1,200 or so inquiries for data every month. People realize that we have a useful product."
Mr. Grant did concede, however, that the center would like to provide data on a more timely basis. He also said that "things are tight" as a result of budget reductions.
"We've lost almost one-fourth of our personnel in the last few years," he said. "Although we're operating with a limited staff, we're operating more efficiently."
Emerson J. Elliott, who replaced Ms. Eldridge as administrator of the center, was unavailable for comment on the proposed evaluation.