The first large-scale international effort to develop a common set of measures for evaluating education systems is moving rapidly toward implementation.
The 24-nation project, undertaken by the Organization for Economic Coordination and Development, a group of Western, industrialized nations, is expected to provide the first comprehensive nation-by-nation comparisons of school systems.
Participants outlined the project’s first phase at the annual meeting of the American Education Research Association in Boston last month.
In that initial 18-month period, they said, representatives of the participating countries reached general agreement on some two dozen indicators to measure the quality of education in five general areas: school enrollment, educational outcomes, school processes, resources and costs, and attitudes and expectations.
Over the next two years, participating countries will collect data for the project’s first report, which is expected to be released in 1992.
Jeanne Griffith, deputy commissioner of the U.S. Education Department’s National Center for Education Statistics, said the report will aid school-reform efforts in each nation by providing the first set of data against which schools can be compared.
“International comparisons represent an opportunity to present standards,” she said. “Internally, you have no basis of knowing whether you performed well.”
“When you compare with other countries,” she added, “you understand your own system.”
The OECD indicators project began in 1987, when the U.S. Education Department convened a meeting in Washington to lay out plans for international education comparisons. (See Education Week, Dec. 2, 1987.)
In 1988, a proposal to develop a set of indicators was approved by the governing board of the Center for Educational Research and Innovation, a branch of the OECD, and member countries formed “networks” to work on selecting indicators in the five general areas under study.
The participants agreed that the indicators should measure “not what is immediately measurable, but what we need to measure,” said Ramsay W. Selden, director of the state education-assessment center at the Council of Chief State School Officers. “There was pressure placed on the system to retain its ideals.”
At the same time, said Ms. Griffith, participants wanted to avoid choosing indicators that might prompt school officials to manipulate their system to achieve a higher result.
Eva L. Baker, a member of a separate Congressionally mandated panel that is studying U.S. education indicators, cautioned against rejecting measures that might be manipulated, however.
“What is corruption, and what is reasonable policy?” Ms. Baker asked at the AERA meeting. “If you generate international data, people will want to act upon it.”
The first of the multi-nation networks, working on enrollment, agreed on a set of four elementary and secondary and three higher-education indicators that would provide some measure of educational outcomes and equity, according to Ms. Griffith.
The elementary and secondary indicators include enrollment, secondary-school completion, grade repetition, and attendance. The higher-education statistics include entrance rates, completion rates, and graduation rates in science and engineering.
During the second phase of the project, Ms. Griffith said, participants must wrestle with several problems that “don’t show up until you plug numbers in.”
One problem, she noted, is how to interpret the raw data. She offered as an example the progress over the past decade of grade-repetition rates, which have climbed rapidly in the United States, dropped sharply in Italy, and remained stable in Australia.
While the U.S. data may appear to indicate poor student performance, Ms. Griffith said, they in fact “reflect changing values associated with social promotion and the need to attain levels of expertise before moving on to the next level.”
But, she noted, “the indicators don’t allow supportive information to explain that trend.”
The group developing measures on resources and costs has agreed to collect data on nine indicators that gauge where school revenues come from; total school expenditures; per-pupil and per-teacher costs; and expenditures in relation to other government spending.
The group decided to include as many measures as possible to avoid controversies like the one that erupted here this year over a study comparing U.S. education expenditures with those of other industrialized countries, said Paul D. Planchon, director of the elementary- and secondary-education statistics division of the NCES.
In that study, he noted, researchers from the Economic Policy Institute used expenditures as a proportion of gross national product to show that the United States ranks near the bottom among industrialized nations in school spending. Other studies, however, have used per-pupil expenditures to show the United States near the top of the international rankings.
During the project’s second phase, Mr. Planchon said, the resources group must work out several technical issues to ensure comparability of data. For example, he pointed out, the United States includes school libraries and transportation in education costs, while in Europe such services are “not typically provided by education institutions.”
The three other working groups have thus far made less progress toward an agreement on which indicators to include.
The group developing measures of school processes has tentatively agreed to collect data on five indicators: the percentage of the adult population working in schools, the pupil-adult ratio, the amount of schooling time, time on task in schools, and curriculum coverage.
These indicators were chosen because research has shown they bear some relation to student performance, said Alan Gibson, an inspector for Her Majesty’s Department of Education and Science in the United Kingdom.
The network on student-outcomes measures is considering developing indicators on secondary-school completion rates, student achievement, and postsecondary employment and education, according to David A. Sweet, a researcher in the U.S. Education Department’s office of research.
To provide a more detailed picture of outcomes, he said, the group is also considering including descriptive information along with the data.
The group developing measures of attitudes and expectations has perhaps the most work yet to do, according to Isabelle Delfau, project director at the OECD But it has agreed, she said, that public attitudes toward school are an important indicator of school quality.
“Debates in education are no longer reserved to the professional few,” Ms. Delfau said. “They are gradually moving out to the general public and the business world.”
