Character education has grown in popularity among educators and parents alike, but the largest federal study of schoolwide programs to date has found that, for the most part, they don’t produce any improvements in student behavior or academic performance.
The Institute of Education Sciences, the U.S. Department of Education’s research arm, gauged the effects of seven typical schoolwide programs from across the country: the Academic and Behavioral Competencies Program of the Center for Children and Families at the University at Buffalo, in New York; the Competence Support Program of the University of North Carolina-Chapel Hill School of Social Work; the Nashville-based Love in a Big World; the Twin Falls, Idaho-based Positive Action; Promoting Alternative Thinking Strategies (PATHS), run by the Channing Bete Co. of South Deerfield, Mass.; the 4Rs (Reading, Writing, Respect, and Resolution) program, operated by the Morningside Center for Teaching Social Responsibility in New York City; and Second Step, run by the Seattle-based Committee for Children.
Researchers from the department’s contractor, the Princeton, N.J.-based Mathematica Policy Research, Inc., randomly assigned 84 schools in six states to receive one of the programs or not. They studied the implementation and outcomes for the schools and more than 6,000 students from the start of 3rd grade to the end of 5th grade. At the end of each year, researchers looked at the effects of the programs, both overall and as individual programs, on 20 indicators relating to social and emotional competence, academics, behavior and perceptions of the school climate. They analyzed the results both for students overall and for four subgroups: gender; students with different initial risk levels; students who had been in the program from the beginning versus newcomers; and students in participating schools with good or poor fidelity to the chosen program.
Finally, at the end of the study, researchers compared the three-year growth of students on those character indicators to the growth of students in the control schools, some of which had their own character-related activities.
Overall, the study found that the schools taking part in the intervention significantly increased their use of character-development instruction and activities. For example, during the three years, 68 percent to 72 percent of teachers in the schools using one of the programs reported using a related activity to address a school character education goal, compared to only 20 percent to 36 percent of teachers in the control group schools. However, the programs did not improve the use of schoolwide social-development strategies or teachers’ attitudes and their individual practices related to character building, such as modeling polite behavior or enlisting students in decision-making.
The programs improved teachers’ sense that students supported them during the first two years of the study, but this effect fell off by the third year. Aside from that, the individual programs showed a mixed bag of results among the 20 indicators that shifted from good to bad from year to year, and none proved significant in the overall findings. In the end, researchers found no evidence that the programs, taken individually or together, improved students’ behavior, academic performance or gains, or their perceptions of the school climate. And the results were no better for schools with better implementation.
“The consistency of the findings was surprising,” said Allen Ruby, the co-author of the report and the associate commissioner for policy and systems at the IES’s National Center for Education Research.
“This is one study, so people shouldn’t just say, ‘We’re done, let’s move on,’ … but they should be challenged to say, ‘OK, let’s look at this in different ways. Let’s look at the data.’”
Program advocates and local investigators likewise voiced surprise and disappointment, in part because some of the programs showed benefits in previous studies. For example, a What Works Clearinghouse-approved study of Positive Action found the program improved both behavior and academic performance of students in Hawaii.
“I think it would be fair to say the investigators were more frustrated than anything else” by the lack of results in the national study, said Brian R. Flay, professor of health and human sciences at Oregon State University. He led the research team that implemented the Positive Action program in Illinois schools and is married to its founder, Carol G. Allred. “We’re going to face some difficulty, because some people are going to see this report and think character education doesn’t work, and that would be the wrong conclusion to come to, based on one report. I think it gives a false sense of failure based on limited analysis.”
Linda McKay, a state and federal character education advocate and a former senior adviser for the Education Department’s character education grant program, said the IES report “really supports what I saw emerging in the research grants for what was effective in character education … that in order for character education to be effective, it really has to be a schoolwide process, a total school framework based on faculty, students, and parents.” However, she said she would have liked more detail on how students, teachers, and parents interacted in the programs, which could shed light on how to improve social-learning initiatives.
Mr. Ruby agreed. “It seemed not finding strong impacts among some of the programs makes you want people to take that step back and say how are these programs supposed to work and what steps do we need to do to make them work?” he said.
Stepping Back
Love in a Big World is one program doing such self-reflection. The Nashville-based program involves 30 weeks of teacher-led reading, writing, and discussion based on specific character traits such as honesty, responsibility, and self-control, along with assemblies, service projects, and other schoolwide activities. The IES study found that during the first year, the program improved students’ altruism and support for their teachers, but by the third year those effects were gone and the program had a detrimental impact on students’ engagement with learning, feelings of safety and positive orientation to school. Overall, the effects were a wash.
The release of the report coincides with a recession-related drought in funding for the program, and founder Tamara Fyke said she has put it into a “holding pattern” while she rethinks its design and completes a master’s degree in education at Vanderbilt University.
As a result of the study, Ms. Fyke said she may refocus the program on media- and Internet-based activities, rather than school initiatives. “I’m of course disappointed, but the whole experience, it was difficult because of how rigorous it was, but it was extremely beneficial,” Ms. Fyke said. “A lot of what I was doing was intuitive, so everything is very valid, but I need to present it differently, with the educational theories to back it up more explicit.”
Researchers Disagree
Yet some local investigators for the programs argued the study missed some program benefits, and some, such as Positive Action and PATHS, expect to publish program-specific studies during the next year.
Mr. Flay, who since has tracked students in the Positive Action program through 8th grade, said research due out later this year has found lower rates of bullying and substance abuse among students who have remained in the program. He argued, “In these inner-city, high-risk, high-poverty neighborhoods, it takes a while for the effects to become significant.”
Joseph Mazzola, the executive director of the Washington, D.C.-based Character Education Partnership, which studies best practices in civic and character education, agreed. “To really see lasting results that I would consider transformative, it would be three years or more,” Mr. Mazzola said. “It’s not an easy thing to do, and it takes committed leadership and parent involvement. It’s not a plug-in-and-go kind of thing.”
Mr. Flay and Mark T. Greenberg, the director of the Prevention Research Center at Pennsylvania State University, in University Park, Pa., who also studied the PATHS program, said the student samples may have obscured results.
“This sort of data is pretty complicated, because these are universal interventions,” Mr. Greenberg said. “If 80 percent of students don’t have behavior problems to begin with, it’s harder to see effects. I don’t think [the Mathematica researchers] were very sensitive to understanding the data in these analyses.”
Yet Mr. Ruby said all local research teams knew and agreed to the sample pool and research design in advance, and the analyses were approved by two separate and independent review teams.
“The analysis group could have looked just at those high-risk kids, but that’s not what they decided. They thought these programs would have an overall effect,” Mr. Ruby said. “If you don’t believe it’s going to have an impact on all these kids, then should we be trying to do comprehensive programs for the entire school population? Saying X percent of my school is having real troubles and I’ll bring in people and social workers to work with those students is a totally different intervention.”
Adding to the Base
Despite the debate, Ms. McKay said the IES report adds to the still-emerging research base for character education. Social and character learning has been hard to define—including, as it does, pieces of ethics, civics, diversity, problem-solving, and social-emotional development, among other topics—and only in the past decade or so has started to get major research attention, Ms. McKay said.
“I think it’s absolutely one of the most critical pieces for education, particularly in high-risk schools,” Ms. McKay said. “If we don’t focus on creating a climate for learning and a classroom culture where students and faculty feel cared for and respected, we won’t get to the academics.”
