A Living Laboratory
|Kentucky's landmark school reform law created rare opportunities for researchers, but so far has yielded few answers.|
In 1989, the Kentucky Supreme Court handed the state's lawmakers and educators a once-in-a-lifetime opportunity. Start over, the justices said. Build a better school system.
The next year, legislators cobbled together a plan from what was then considered the best of education's cutting-edge ideas. They called for student-performance measures that included projects, portfolios, and tests with open-ended questions. They set new academic standards and goals and added rewards and sanctions to hold schools to the new targets.
Schools were also asked to create multiage classes in the primary grades, give teachers more on-the-job training, and set up preschool programs for poor 4-year-olds. In turn, they received more authority and more-equitable funding.
At that time, a few states were tinkering with some of those strategies. But no other state was trying all of them at once.
All of that activity has transformed a poor, mostly rural state with lagging schools into a living laboratory for state-of-the-art educational innovations. Researchers from around the country--as well as from within the state--headed for Kentucky to examine everything from preschool programs to teacher training initiatives.
In addition to its grand scale, Kentucky's reform effort offers researchers the added bonus of longevity. Despite some fine-tuning, the major pieces have largely been in place for seven years. "Most statewide efforts don't last more than three years," says Roger Pankratz, the executive director of the Kentucky Institute for Education Research, based in Frankfort.
As a result, school administrators and policymakers have amassed an impressive stack of studies on the state's progress. The research has provided insights on a wide range of issues--school finance, accountability, pay incentives for teachers, and strategies for improving failing schools, to name a few.
As more states have sought to create comprehensive and integrated approaches to education policy, many have looked to Kentucky for guidance.
But, seven years and more than 500 studies into the program, researchers still can't say for sure whether Kentucky's reforms have made a real difference in the classroom.
"We don't have great information on how kids' learning is different," said Jane Clark Lindle, an associate professor of education at the University of Kentucky and a co-director of the University of Kentucky/University of Louisville Joint Center for the Study of Educational Policy. "And that's the critical issue."
Part of the problem is that the very thing that has made Kentucky attractive to researchers has also made it difficult to study effectively: A lot is going on at once. In the same schools where educators are flexing newfound autonomy, students grapple with unfamiliar tests and the use of portfolios. And, in a state where all the schools are changing, it's hard to find control groups for comparison.
"The disadvantage from a research point of view is trying to tease out which factors were responsible for the changes that occurred," observed Connie Bridge, a professor of education at North Central College in Naperville, Ill. "Was it the assessment, or were teachers getting more staff development?"
Researchers drew a few conclusions about the reforms early on. For example, studies show that the new funding system has successfully halved the gap between Kentucky's highest-spending districts and its lowest-spending.
Surveys also show that most schools have elected governing councils of educators and parents to decide everything from which textbooks to buy to who will be the principal--decisions once left to superintendents and school boards.
The state has fared less well in its effort to establish mixed-age classrooms in grades K-3, a recommendation that is strongly encouraged by regulations governing the reforms but is no longer mandated. Surveys show that only 10 percent of teachers group children from three or more age levels in their classrooms; most group children from just two grades. And a smattering of studies shows that the numbers of teachers assigned to mixed-age classrooms decreased further once the mandate was removed.
As for student achievement, the picture is mixed. Scores on the new assessment have for the most part risen since 1992-93, but most of the movement has come at the lower end of the scoring scale. The state has had a tougher time nudging students to reach "proficient" and "distinguished" levels of performance. And elementary students tend to outperform their high school and middle school counterparts.
|Over the years, some investigators have questioned whether the gains in the early years were real.|
The picture changed somewhat this school year. The latest round of test results shows gains at all levels of schooling in all subjects. And, in the two areas with the biggest gains--reading and mathematics--fewer students scored at the "novice" level and more reached the proficient and distinguished levels.
Still, over the years, some investigators and educators have questioned whether the gains in the early years of the program were real.
As the linchpin of the reform effort, the state testing program--known as the Kentucky Instructional Results Information System, or KIRIS--was designed to be far-reaching. The idea was to develop tests worth teaching to. Thus, rather than focus on basic skills as many standardized tests do, Kentucky's assessments emphasize writing, hands-on tasks, and open-ended test questions.
Students also compile portfolios of their best work in math and writing, though currently only the writing portfolios are scored.
To ensure that the tests would drive changes in classroom practice, lawmakers built in a system that rewards schools that improve and penalizes those where scores either remain steady or decline. A "distinguished educator" is assigned to evaluate and assist each school in the latter category, and the entire staff is placed on probation.
The high stakes attached to KIRIS have made it one of the most heavily scrutinized pieces of Kentucky's reforms. "People are interested in what happens in Kentucky's accountability system even if they never set foot in Kentucky," said Dan Koretz, a Washington-based researcher with the RAND Corp. who has studied the testing system.
Over the years, several researchers have criticized technical aspects of the program, such as the validity of the scores on writing portfolios. And they have questioned why yearly gains on the state tests were not reflected in some other tests, such as the ACT college-entrance exam.
But a more recent study by Anthony Nitko, a University of Pittsburgh researcher, concludes that the open-ended questions on the state tests are, for the most part, measuring what they're supposed to: how well schools cover the 37 academic expectations set by the state. Many commercially available standardized tests do not.
A team of University of Kentucky researchers led by education professor Joseph Petrosko also studied 19 elementary schools and found that schools with high KIRIS scores tended to be those that had also put in place the reforms called for by the 1990 law.