Washington
The Department of Education unveiled an ambitious assessment agenda for Title I last week that reflects the compensatory-education program’s new emphasis on academic standards and large-scale reform.
Previous assessment teams that have examined the program, which was named Chapter 1 from 1981 to 1994, have issued interim reports that were mainly descriptive. But “Mapping Out the National Assessment of Title I: The Interim Report” is primarily a blue-print for what would be the most far-reaching effort to date to weigh the effectiveness of the largest federal program in K-12 education.
The strategy is designed to gauge Title I’s ability to improve the performance of the poor and disadvantaged students it is designed to serve by teaching them to challenging academic standards. Because of changes made to the program during its last reauthorization in 1994, the evaluation will also look at Title I in the context of broader state and district reforms and in conjunction with other federal education programs.
“We’re trying to get away from some of the traditional ways of doing evaluations,” Undersecretary of Education Marshall S. Smith said in an interview.
In addition to “posing challenges to the traditional evaluation paradigms,” he said, the evaluation plan provides “a new way of looking at the role of federal, state, and local governments.”
The interim report, which was delivered to members of Congress last week, outlines 11 studies that will make up the assessment. A long-awaited longitudinal study of student achievement in the previous Chapter 1 program is also to be completed in 1997. (See box, page 27.)
The report also sets the criteria by which the program’s facets will be judged. For example, state and district efforts to reform Title I curricula and instruction will be measured by such indicators as the exposure of Title I students to “challenging subject matter,” the effectiveness of teaching practices, and the extent to which officials have made use of consolidated program planning.
A final report is due in January 1998, but the department plans to provide “indicators of early implementation and interim progress” before then.
Warning Call
An independent panel charged with monitoring the implementation and evaluation of the new Title I program praised the assessment agenda but sounded some warnings as well.
In a letter to the congressional leadership and members of the House and Senate education committees, the Independent Review Panel for the National Assessment of Title I said that adequate data on the effectiveness of the program will not be available from all states or districts by the time the final report is due.
The panel noted that state standards in mathematics and reading, which are to guide Title I instruction under the new law, are not required to be in place until the 1997-98 school year; that states are allowed up to the full five years the 1994 reauthorization will be in force to create new assessments; and that transitional assessments are not required to report performance data on students receiving Title I services.
“It will be impossible to address the most critical questions about the law’s implementation and effectiveness,” said the letter, which was signed by Christopher T. Cross, the panel’s chairman and the president of the Washington-based Council for Basic Education.
One longitudinal study is included in the department’s Title I evaluation plans, but a final report is not scheduled for release until 2000.
Department officials and the review panel also expressed concern that Congress may not appropriate the money needed to produce the planned series of studies.
The Clinton administration requested $11 million for Title I evaluation in fiscal 1996, but the appropriations bill pending in the Senate would provide only $3.4 million and the companion bill approved by the House includes no funding at all.
Mr. Cross, who served as the Education Department’s assistant secretary for educational research and improvement during the Bush administration, said Congress must appropriate about twice the Senate allocation to allow a meaningful evaluation.
The interim report notes that the 1993 National Assessment of Chapter 1 and “Prospects,” the longitudinal study of Chapter 1 that is to be completed in 1997, were funded at $6 million annually. (See Education Week, Nov. 24, 1993.)
Effectiveness Questioned
The interim report provided little new data, but does include some new findings from the ongoing “Prospects” study. The researchers found:
- First-grade students who participated in Chapter 1 in 1991 scored an average of 42 points lower on vocabulary tests than other students. By 1994, when these students were in the 4th grade, the gap rose to 44 points. Their reading-comprehension scores showed a gap of 38 points in 1991, and rose to 60 points in 1994.
- The cohort of students who were in 3rd grade in 1991 showed a different pattern. Chapter 1 students in this age group were outscored by their peers by 40 points on vocabulary measures in 1991, compared with 45 points in 1994. But the difference in comprehension scores was 56 points in 1991 and 44 points in 1994.
The report says similar patterns were found for mathematics.
The data, “while disheartening, provide clear benchmarks against which to measure Title I efforts to raise student performance,” the report says.
These results are especially troubling for the program’s proponents in light of congressional Republicans’ increasingly vocal skepticism about its effectiveness. Both House and Senate appropriators have called for dramatic cuts in Title I. Even some longtime supporters have been publicly casting doubt upon the program. (See Education Week, Nov. 8, 1995.)
Education Department officials recognize both the importance of achievement data and the short time frame in which it must be collected. The interim report notes that they plan to release data throughout the evaluation.
“Student performance will need to be assessed incrementally over the long term,” the report said. “We should not expect achievement gains to occur overnight, and not without sustained policy and programmatic changes at the federal, state, local, and school levels.”
Copies of the interim report are available from the U.S. Department of Education, Office of Planning and Evaluation Service, 600 Independence Ave., S.W., Room 4168, Washington, D.C. 20202; telephone, 1-800-USA-LEARN.
