A Living Laboratory
|The law requires districts to provide teachers with at least four days of in-service training.|
Such positive findings, which have come out only in the past year, still may not be enough to sway teachers, who report in survey after survey that they do not trust the test results. Many believe students are scoring higher because they are becoming more familiar with the tests.
Emphasis on Writing
There is less doubt, on the other hand, that the tests have changed the mix of activities that take place in the classroom.
Ms. Bridge, of North Central College, studied instructional practices at four Kentucky elementary schools in 1982. She returned to the same schools in 1995, five years into the reforms, to see whether teaching had changed.
She found that in the grades she studied--1st, 3rd, 4th, and 5th--teachers were doing 1.2 to two times as much writing with their students as they did before the reforms. Students were writing expository essays, newspaper articles, stories, paragraphs, and essay responses of the type called for in the state testing system.
"If you think about doubling the amount of time that teachers spend [on] writing, I think that's pretty significant," Ms. Bridge said recently. A handful of other studies echo her findings.
But teachers also complain that all the writing, rewriting, and conferring over writing involved in the new approaches eats up too much classroom time. "Teachers in Kentucky felt they had to give up important material," said Mr. Koretz, who surveyed 500 Kentucky teachers and principals during the 1994-95 school year.
What they have sacrificed, teachers typically report in surveys, is instruction in basic computational and language skills.
To test whether students were indeed losing out on those skills, the state hired the Human Resources Research Organization, a Radcliff, Ky.-based group, to compare examples of student writing from 1993 and 1996. The researchers analyzed the samples for spelling, capitalization, punctuation, and subject-verb agreement.
"The long story is that 4th grade girls' performance essentially remained flat, and the boys improved," said Gene Hoffman, the manager of the group's center for learning, evaluation, and assessment research.
Success With Preschool
The 1990 law called upon districts to set up preschool programs for at-risk 4-year-olds and for children ages 3 and 4 with disabilities.
A year later, researchers began tracking groups of children who entered the program and comparing them with other youngsters who were eligible for preschool but whose parents decided against enrolling them. The sample now includes about 3,000 children, the oldest of whom are in 5th grade, according to Mary Louise Hemmerter, an associate education professor at the University of Kentucky who is working on that project.
"What we are finding is that these children make progress," she said. "In kindergarten, teachers rate them as prepared as children from higher-income families and more prepared than children who could've gone through the preschool programs but did not."
Over time, the study also found, the former preschool pupils kept pace with their better-off peers. And those students' grades did not drop off in 3rd grade--the point at which studies of other preschool programs suggest that achievement gains made in the early years start to fade.
Training Focus Studied
In the area of professional development for teachers, another bulwark of the reform program, the research evidence is more mixed. The law requires districts to submit their plans for professional development to the state and to provide teachers with at least four days of in-service training. Schools also received extra money to pay for the added training.
But Thomas Corcoran, a co-director of the Consortium for Policy Research in Education, a national organization based at the University of Pennsylvania, said schools have not made the wisest use of those opportunities. He and two colleagues studied 20 schools noted for doing "interesting things" in professional development. They reviewed those schools' plans and talked to heads of their professional-development committees.
"The focus in most places was on the short term," Mr. Corcoran said. "It was, 'What can we do with the money we have and the four days we have to jack up our KIRIS scores?'" As a result, activities tended to focus on pedagogical techniques, such as cooperative learning, and new assessment strategies. Little attention was paid to deepening teachers' understanding of the subjects they teach.
On the other hand, he added, "the good news is that there's more professional-development activity and more of it is schoolwide, and there's really an effort to get some practical payoff out of it."
"Even though the definitive word on KERA's success or failure is
yet to come, no one in Kentucky has advocated throwing out the
The reform package also called upon schools to establish extended
programs to offer help to struggling students before and after school
and during the summer. Schools with a high proportion of poor students
were also required to set up resource centers where families could
receive help obtaining eyeglasses or medical referrals for their
children. And studies on those programs suggest they are up and running
and doing their jobs.
Broader Research Needed
While such research on individual provisions of the reform law is useful, Ms. Lindle of the University of Kentucky says, overall the studies on its impact have been mostly small, fragmented, and not nearly as numerous as researchers and state officials might have hoped. "Here we've got a massive reform system and no real window into it," she said.
Research gained a higher profile after Wilmer S. Cody became the state's second appointed superintendent in 1995. He named William White last year to be the first research manager for the education department since it was re-created in 1990. The department is also devising a peer-review process to raise the quality of the studies it pays for.
Mr. White says the time has come to shift the focus away from studies that look just at the reform program's "inputs."
"We are at the point that most of the pieces are in place, and we need to devote more time to evaluating the effectiveness of programs," he said. "For example, we know a lot about what kinds of technology we have in place and where, but we don't know a lot about how technology is actually being used in the classroom by our teachers." The department is in the process of commissioning studies to answer that question.
In a similar vein, the Kentucky Institute for Education Research, which Mr. Pankratz heads, is re-examining its mission. The organization, with support from the Annie E. Casey Foundation, has bankrolled much of the research on the reform program to date. The institute's funding is scheduled to run out before the close of the century, prompting its leaders to review what role it will play in future research.
Even though the definitive word on KERA's success or failure is yet to come, Mr. Pankratz says, no one in Kentucky has advocated throwing out the reforms.
"I think we have been effective in trying to convince legislators and others to improve what we have rather than starting over," he said. "Whether we're going to be fortunate enough to learn from our mistakes and successes--that's not been answered."
PHOTO: Patricia Kannapel of the Appalachia
Educational Laboratory is a co-director of a research team that since
1991 has conducted fieldwork on the effects of Kentucky's 1990 reform
law on four districts.
Roger Pankratz, the director of the Kentucky
Institute for Education Research, says the longevity of the reforms has
allowed time for study.