Improvement Plan For Chapter 1 Seen Needing Overhaul
WASHINGTON--As the Chapter 1 "program improvement'' initiative enters its third year, many observers have concluded that the controversial program is not working as intended. Educators, researchers, and advocates familiar with Chapter 1 agree that lawmakers should retain the accountability concept when the Congress reassesses Chapter 1 next year, but that the improvement process needs a substantial overhaul.
"I'm not prepared to say it's a failure,'' said Cynthia Brown, the director of the resource center on educational equity at the Council of Chief State School Officers and a key member of a Chapter 1 study commission established by the council.
"The idea is right--we need accountability based on student outcomes,'' she said. "But there are big problems with how it's implemented and particularly with the assessment system.''
The program-improvement initiative, created when Chapter 1 was reauthorized in 1988, requires schools to enter into a formal improvement process if their Chapter 1 students do not meet minimum achievement standards. State officials are authorized to intervene if improvement does not occur after a year of improvement efforts.
Observers agree that the process has served as a catalyst for change in many schools, as educators have been forced to evaluate their Chapter 1 programs for the first time in the remedial program's more than quarter-century history.
The problem, observers say, is that the minimum standards are sufficiently low and the rules sufficiently lenient that many schools whose students do not perform at acceptable levels can slip through the process without making substantial changes.
"The tools in the law are potentially very powerful ones, but weaknesses in the process have undermined their effectiveness,'' a group of researchers studying the implementation of the 1988 Chapter 1 amendments for the Education Department recently concluded.
"Few districts are undertaking program-improvement activities of any magnitude,'' states the report by Abt Associates of Cambridge, Mass.
Increase in Participation
About 20 percent of Chapter 1 schools are currently engaged in some stage of the program-improvement process, according to new statistics gathered by the Education Department. That is more than twice as many schools as were involved in the first year, during the 1989-90 school year.
Observers attribute the boost in participation to increases in minimum standards and better monitoring by state and local officials as they become familiar with the system.
However, new statistics from schools identified in the first two years indicate that relatively few schools are undergoing program-improvement efforts for a long period of time.
The law requires schools to use a minimum national test-score standard and higher or additional standards set at the state and local levels to chart the performance of Chapter 1 students annually.
In schools that fail to meet the standards, educators must draft improvement plans, which can be implemented in the same year a school is targeted or in the following year. If a school fails to meet its goals after a year of improvement efforts, its staff, working in conjunction with state officials, must draft a new plan.
Of the 4,964 schools identified in the first year of the program, 2,016 remained in the system for a second year. Of the 6,931 schools newly identified in 1990-91, 3,530--again, about half--were retained for a second year.
But of the original group of targeted schools, only 644 are entering their third year of improvement efforts. Only about 1,400 schools are currently working on joint plans with state officials.
Observers say the relatively small number of schools involved in the second and third years of program improvement is the result not of successful reform initiatives, but of the fickleness of test scores, low minimum standards, weaknesses in the system, and sometimes pointed efforts by educators to remove their schools from it.
"I think many schools became immersed in the issue of identification, delayed implementation [of improvement plans], and saw [the improvement process] as something to get out of,'' said Mary Jean LeTendre, the director of compensatory-education programs for the Education Department. "They're more concerned with whether their kids test right than with seriously looking at how to do a better job.''
"I would conclude that program improvement has not reached the great potential it has for leveraging change,'' she said.
Standard Too Low?
Chapter 1 advocates have complained from the start that the minimum national standard to avoid being targeted for program improvement, set in the Education Department's regulations, is much too low. It targets schools where the aggregate achievement of Chapter 1 students has remained stagnant or actually declined, as measured in "normal curve equivalents.''
The N.C.E. scale, developed for Chapter 1, is designed to be applicable to results from a variety of standardized tests. It measures a student's progress relative to his peers, and gaining one N.C.E. is very roughly equivalent to gaining one percentile point on a typical 100-point scale. Theoretically, a student who maintains the same N.C.E. score has learned during the intervening year, but has not gained relative to other students.
The average gain for Chapter 1 students nationally is about 3 N.C.E.'s in reading and 4 in math, while successful programs boast average gains of 10 or more N.C.E.'s.
The minimum requirement for escaping program improvement is expressed as an N.C.E. average greater than zero. While federal officials have joined advocates in urging state and local officials to set higher standards--as well as establish additional standards, such as grades or promotion rates--the minimum test-score standard has remained the bottom line in many places.
Ms. LeTendre of the Education Department said she was "extremely pleased'' that 26 states set a higher standard than the minimum this year, compared with 11 in the first year of the process and 19 in the second.
But that means as many states and territories are still using the minimum mark. Only 2 states use a standard higher than 2 N.C.E..'s--Oregon and Wisconsin, which both set a 3-N.C.E.. standard.
In states using the minimum standard, Ms. LeTendre said, an average of 15 percent of Chapter 1 schools are targeted for improvement. Those with a 1-N.C.E. standard have an average of 21 percent, and it rises to 38 percent of schools in states where the standard is 2 N.C.E..'s or higher.
Higher Standards Sought
Ms. LeTendre said many state officials do not set higher standards because they fear they would not have the staff and resources to help the larger number of schools that would be targeted.
The Congress only provided $5.8 million for program improvement in the 1989-90 school year, and the Abt report found that the median size of a grant to a school district was only $1,000. In 1990-91, when $12.5 million was available, the median grant doubled to $2,000. Only 19 percent of districts receiving program-improvement grants received $10,000 or more.
Milton Matthews, the Chapter 1 coordinator for Mississippi and the president of the National Association of State Chapter 1 Coordinators, said he favors higher standards, but noted that state officials "face a lot of pressure'' from school districts.
"With a child that is below average to begin with,'' he said, "to say you must sustain N.C.E.. growth, high expectations aside, you're almost moving them into a cannot-do-it situation.''
But child advocates have little sympathy for such arguments.
"I think it's appalling that the highest standard is still 3 N.C.E..'s,'' said Paul Weckstein, a lawyer at the Center for Law and Education who represents the National Association of Title I/Chapter 1 Parents.
Not only is that standard too low in principle, critics say, but it also allows marginal schools to post high enough gains to "test out'' of program improvement when they have done little to change their programs.
"With that low a standard,'' Mr. Matthews acknowledged, "you could show enough [gain] to test out just due to flukes in testing.''
Observers cite anecdotal evidence that many schools leave program improvement without even implementing their plans, which they can do if their test scores improve enough during the planning year.
"Across the districts visited, over half of the schools identified as in need of improvement had tested out, yet very few had initiated any programmatic changes,'' states the Abt report, which relied on surveys and site visits to 27 districts and 9 state agencies.
Of 25 districts where schools had been targeted for improvement, the Abt researchers found that about two-thirds "contained schools that took the planning process seriously.''
But they also found three districts that took no action at all and three others where energies were focused on getting schools out of the improvement process rather than on improving the content of the Chapter 1 program.
The report cites as an example a district where kindergarten classes did not meet their targets; in response, officials "removed Chapter 1 funding from that grade level and replaced it with more user-friendly state compensatory money.'' Other schools changed their tests, the researchers said, and one state's "strategy for program improvement was to align the Chapter 1 curriculum with the test.''
Many observers propose attacking these problems by raising minimum standards and requiring targeted schools to stay in program improvement for several years. The Abt report recommends a three-year period, and also suggests that statistical anomalies could be reduced by evaluating schools' progress over several years, rather than from one year to the next.
But the researchers, like other Chapter 1 experts, argue that the improvement process's heavy reliance on norm-referenced, standardized testing is a serious flaw in itself. They assert that it is unclear whether such tests are valid measures of improvement over time.
Problems With Targeting
The report also concludes that lukewarm response to program improvement is directly related to educators' belief that the targeting process is arbitrary.
In some districts, the report notes, "nationally recognized schools'' are identified for program improvement, while "schools that seemed prime candidates in the eyes of school and district people were bypassed.''
The researchers point out, as have many Chapter 1 educators, that a program can qualify for program improvement based on "a small number of test questions answered right or wrong by one student.'' Also, they note, because the Chapter 1 population tends to be highly mobile and the most successful students graduate from the program, the students taking tests in two consecutive years are not the same students.
In addition, a scale measuring only test-score gains does not discriminate between schools with widely disparate circumstances.
"A school can go from the 10th to the 13th percentile, which is clearly inadequate, and not be identified, while the school whose kids stay at the 50th percentile is identified,'' said Phyllis McClure, the director of education programs at the NAACP Legal Defense and Educational Fund and a member of two Chapter 1 study committees.
In a recent article, Robert E. Slavin and Nancy A. Madden of the Center for Research on Effective Schooling for Disadvantaged Students at Johns Hopkins University show how the use of test-score gains effectively penalizes schools that pursue early-intervention strategies and rewards those with high grade-retention rates.
A school that identifies and helps low-achieving students at an early age can set an N.C.E..-gain benchmark that is difficult to exceed each year, they argue, while a school whose students have fallen further behind begins with lower pre-test scores.
Meanwhile, they write in the article, which was published in the latest issue of Educational Evaluation and Policy Analysis, schools that retain low achievers have an advantage if those students take the same grade-level test two years in a row.
In another article in the same publication, Philadelphia's superintendent of schools, Constance E. Clayton, reports that schools in her district have been forced to divert resources to grade levels and subjects showing declines in test scores--even though both pre- and post-test scores for those programs exceeded those of programs that posted gains.
Observers also note that standardized tests do not measure higher-order thinking skills or students' progress in the regular school program--both concepts that were included in the 1988 amendments.
"What's needed is not simply to have everybody in program improvement,'' Mr. Weckstein of the Center for Law and Education said. "The standard must measure substantial progress toward the skills all kids need to learn, as the statute requires.''
Calls for New Design
Chapter 1 advocates are virtually unanimous in asserting that a new way to assess the program's impact must be found.
"We need to design a whole new assessment system for Chapter 1,'' Ms. Brown of the Council of Chief State School Officers said. "The whole system is basically fraudulent.''
"We have to look at using more than one sole measure,'' Ms. LeTendre of the Education Department said. "We have to look at achievement in the regular program, grade-level performance, and advanced skills. We have to find ways to measure those outcomes.''
Coordination between Chapter 1 and the regular program must also be assessed, she added.
"Where we really see program improvement working is where it serves as a catalyst for change in the whole program,'' Ms. LeTendre said.
However, observers unanimously agreed that the program-improvement concept should be refined, rather than scrapped, when the Congress reauthorizes Chapter 1 next year. They point out that the process has spurred many schools to examine their programs in a way they never did before.
"In that respect, it is a big success,'' Mr. Matthews of the
National Association of State Chapter 1 Coordinators said. "It may not
be what some people envisioned, but I think we are making a change.
We're not batting 1.000, but we're not striking out,