In one district, seeing survey data about school climate and students’ self-perception of social and emotional strengths motivated educators to change their practices, a new report concludes. And that was true even though the survey results weren’t used for high-stakes purposes, like teacher evaluations.
What’s not known is whether the adjustments educators made effectively addressed the issues they were concerned about or whether they will move the needle on future survey results, says the report by Future Ed at Georgetown University.
The report comes at a time when researchers are exploring whether it’s possible to consistently and reliably measure social-emotional learning by asking students about their own strengths in areas like relationship skills. Some have criticized such measures, including those explored in the report, saying they are prone to flawed responses or being “gamed” by educators seeking desirable results.
The report looks at the experiences of the Fresno Unified School District, part of the CORE districts, a group of California school systems that regularly survey students about social-emotional learning competencies, like social awareness. The districts also survey teachers, students, and parents about school climate issues, like safety.
“There’s a good deal of research yet to be done, but what struck me was that, even in a low-stakes environment, just seeing data on social-emotional issues and school culture and climate conditions really has proven to be a substantial catalyst for educators on all levels of the district,” said Future Ed director Thomas Toch, who co-authored the report.
Those CORE district’s measures were originally developed as part of a unique waiver from the requirements of No Child Left Behind. That federal law has since been replaced, and the waiver has lapsed, but the districts continue to administer the surveys.
“Fresno educators told us that merely by administering the annual surveys to students, educators, and parents and conveying the results to schools, the CORE districts have signaled the importance of social-emotional contributors to student success and have galvanized educators to act on problems the surveys surface,” the Future Ed report says.
A graphic details the school climate and social-emotional learning issues the CORE surveys ask about.
Authors detailed the reaction to survey results at one Fresno high school:
Only a third of students believed they could master the toughest topics in their classes. Less than half responded positively to the question, 'I feel like I'm part of this school.' Parents felt Hoover was far safer than educators did. These problems, [the principal] and her colleagues believed, were contributing to the school's troubling academic performance."
Staff responded with day-long professional-development sessions, working with a school climate coach from the district’s central office to help unpack their students’ responses to the school and determine how they should respond. They learned new strategies to draw out students’ personal experiences in classroom discussions, hoping to build their sense of belonging. And they discussed ways to help students try multiple strategies for solving stubborn problems.
How those efforts affected students’ responses won’t be known until Fresno gets responses to this year’s surveys. And Toch says it’s a challenge for any district that collects such measurements to ensure they have developed proven improvement strategies that can effectively address weak points they may uncover.
Reliability Questions
The report touches on concerns about the reliability of measurements, discussing the controversy over the CORE district’s surveys and external efforts to validate them. To address some concerns, the survey results aren’t reported at a student or classroom level, the report says. Presented by grade level and by subpopulations, like race or ethnicity, the data are less specific than those collected for accountability purposes.
Social-emotional learning researchers are generally more comfortable measuring school climate than asking students about their own personal traits, which can lead to inconsistent answers depending on the child and their understanding of the question. The Aspen Institute National Commission on Social, Emotional, and Academic Development, recently recommended schools set SEL benchmarks or standards, but it recommended against assessment of those skills.
The Fresno report notes a “caution flag” about how the surveys are administered. One principal, for example, told authors he once explained the survey questions over the intercom, “a well-intentioned step that could introduce bias into survey responses.”
“And we don’t know if parents, students, and educators would continue to give genuine responses year after year, or if social desirability bias or other challenges might creep into the surveys, threatening their reliability, especially after schools like Hoover make major investments in improving school culture and strengthening student engagement,” the report says.
Among Future Ed’s recommendations for districts interested in measuring SEL or school climate:
- Surveys should be focused and questions should be limited so they don’t overwhelm respondents.
- Report results by subpopulations to identify differences in their perceptions or in how they interpret questions.
- Report aggregate results at grade level but not at classroom or student level to avoid stigmatizing students because of their responses.
- Don’t use survey results for high-stakes purposes like accountability.
- Focus improvement strategies and don’t overwhelm schools with too many at once.
- Build support in schools and districts to help educators improve school climates and support students.
Improving social-emotional factors should lead to an improvement in academics, Toch said, and a good school climate is not a replacement for strong instruction.
“It should be a both/and and not an either/or,” he said.
Photo: Getty Images
- Moving Beyond Just Academics in Assessing Effectiveness
- Students Help Design Measures of Social-Emotional Skills
- CORE Districts to Share Social-Emotional Measures to Inform Accountability Plans