Special Report
Student Well-Being

Moving Beyond Just Academics in Assessing Effectiveness

By Evie Blad — December 30, 2015 10 min read

As discussions about school accountability begin to focus more intently on factors beyond standardized-test scores, educators and policymakers nationwide are closely watching a group of California school districts—collectively known as the CORE districts—as they rethink how they evaluate school effectiveness.

In 2013, six districts received the only local-level waiver from the U.S. Department of Education when it bypassed California’s state education department to excuse those school systems from some requirements of the outdated No Child Left Behind Act.

In return for the federal flexibility, the districts—including Los Angeles Unified, San Francisco, and Fresno, some of the state’s largest—agreed to create a first-of-its-kind local accountability system that relies on a broad range of indicators in addition to traditional test scores to monitor schools.

Those indicators include suspension rates; school-climate survey responses from parents; and measures of traits related to students’ social development and engagement, like self-management and social awareness.

That work foreshadowed an inclusion of nonacademic factors as a part of school accountability in the Every Student Succeeds Act, a bill to reauthorize the Elementary and Secondary Education Act that was signed into law by President Barack Obama in December. Under ESSA, state accountability systems will be required to include at least one nonacademic indicator. The legislation lists educator engagement, student engagement, and school climate measures as examples, and it leaves the door open for others.

“We need to make sure that our schools’ quality is measured in a way that is much more reflective of the hard work that’s been done,” said Antwan Wilson, the superintendent of the Oakland district, which is also under the CORE waiver, along with Long Beach and Santa Ana. (Three other districts—Garden Grove, Sacramento, and Sanger Unified—are a part of CORE, but do not operate under the waiver.)

Broader Accountability

The accountability system, known as the school quality index, will be maintained by an organization that is governed by the districts’ leaders known as the California Office for Reforming Education, or CORE. (The Sacramento and Sanger Unified districts were originally approved to be a part of the waiver group, but later withdrew.)

Efforts to broaden accountability to include nonacademic factors come with some big questions.

At a time when some influential researchers have cautioned against using measures of noncognitive traits in school accountability, CORE officials are navigating uncharted territory in their ongoing work to determine what student traits should be used in their accountability system, how to measure those traits, and what to do with the results.

Their work could be informative to states as they seek to select what “other indicator” they will use to measure their schools as the federal government steps back from its previously prescriptive model for accountability and devolves more authority to the states in designing their systems.

The process of creating the CORE accountability system has also created excitement among education leaders who see it as a chance to move away from traditional accountability measures, which they say rely too heavily on standardized-test scores and tend to foster a culture of blame among educators rather than an atmosphere of support.

CORE pairs schools that fall to the bottom of the quality index and also rank lowest in academic achievement, known as priority schools, with demographically similar high-achieving schools. The aim is that principals and teachers trade techniques and strategies to improve.

The system also gives every school a “dashboard” of indicators it can monitor throughout the school year.

“The history of accountability systems has been about, ‘What is the thing I need to do to avoid the punishment?’ ” said Noah Bookman, CORE’s chief accountability officer. “We are trying to move the conversation to: ‘What am I learning? What strengths do I need to leverage? What are the challenges I need to address?’ and ‘What do I need to do that?’ ”

How It Works

Beachy Avenue Elementary teacher Cocoro Morimoto displays a card summarizing the Second Step program concepts that she uses with her 4th graders. She believes lessons from the social-emotional learning program have had a great impact on student success.

Key to the CORE districts’ waiver was a promise to reduce the number of students required for a school to be held accountable for a given subpopulation—students from racial- and ethnic-minority groups, English-language learners, low-income students, and special education students—from 100 to 20. That means schools with smaller numbers of students from those populations will be held accountable for their performance for the first time under the new index.

The school-quality-improvement index bases 60 percent of a school’s score on academic factors. Under that category, schools will be judged by their proficiency rates and growth in proficiency rates on the Smarter Balanced test, aligned to the Common Core State Standards.

That yardstick includes the scores of all students combined, as well as the scores of each subpopulation: English-language learners, low-income students, students with disabilities, and students in the lowest-performing racial or ethnic subpopulation at that school.

High schools’ academic-category scores will also include how their four-year, five-year, and six-year cohort graduation rates compare with those of other high schools in CORE districts.

And middle schools’ academic scores will include a “high school readiness” rate, which is the number of 8th graders who meet criteria researchers have linked to a higher likelihood of high school graduation: a GPA of 2.5 or better, an attendance rate of at least 96 percent, no Ds or Fs in English/language arts or math in 8th grade, and no suspensions.

The remaining 40 percent of a school’s score will be a combination of factors related to school climate and students’ noncognitive skills, such as self-management. Current school-level report cards crafted under the index include three factors in this category: rates of chronic absenteeism, suspension and expulsion rates, and the rate at which English-language learners are redesignated as fluent. Beginning in 2016, two other factors will be added to that nonacademic domain: measures of students’ noncognitive skills and the results of student, staff, and parent surveys about school climate and safety.

Social-Emotional Factors

After consulting a growing body of research that links such skills to a higher likelihood of college and career success, the CORE districts identified four student traits to measure and track: social awareness; self-management; self-efficacy, which is the level of confidence an individual has in his or her ability to succeed and to “control ... their own motivation, behavior, and environment;" and growth mindset, which is defined as an understanding that academic skill is not an inherent, fixed trait but one that can grow through effort.

The districts plan to assess those traits through surveys that ask students questions like how frequently they come to class prepared and how much they agree or disagree with statements such as: “Challenging myself won’t make me any smarter” and “I can earn an A in all of my classes.”

CORE districts piloted a longer menu of survey questions with 450,000 students. After refining that list, CORE will conduct surveys at every school this year.

Leaders of the districts say including such factors in the index will help track the effectiveness of social-emotional-learning programs and similar interventions already in place in many of their schools and help them call out good work that may previously have gone unrecognized.

Oakland, for example, is one of eight urban districts piloting a comprehensive social-emotional-learning program in consultation with the Collaborative for Academic, Social, and Emotional Learning, a Chicago-based organization that promotes research, policy, and practice related to such strategies. Other districts have made efforts on a school-by-school basis.

Among those schools is Beachy Avenue Elementary, which is part of the Los Angeles Unified School District. It uses the Second Step social-emotional-learning program to teach regular lessons in subjects such as empathy and anger management alongside its core subjects.

In one such lesson this fall, third-year teacher Cocoro Morimoto showed a video of two young boys describing their interactions at a sleepover—one had a great time, and the other lamented that his friend left his room a mess—and asked her 4th grade class to identify and relate to each boy’s perspective. In the future, she asked, how could the boys handle their interactions differently?

“Empathy!” several students yelled as they remembered the concept all at once.

Such lessons have a significant impact on student’s academic success, Morimoto said in an interview. Before she used the curriculum, she lost valuable instructional time addressing student conflicts that erupted during recess.

“I think they feel more empowered when they are solving problems on their own,” she said, adding that social confidence helps boost students’ confidence with challenging academic content. “I am giving them words for their feelings.”

Even without such measures, Beachy Principal Stephen Bluestein said he knows the social-emotional lessons are working. Since adopting the approach, the school’s office referrals have dropped and individual students’ academic performance has improved, he said, although the school still performs below district averages academically.

“It has indirect and direct positive effects that reflect in reading, writing, and arithmetic,” he said.

Researcher Cautions

But will the CORE districts’ noncognitive measures detect the changed attitudes of students?

Researchers of that topic have warned that those measures should not be used for school accountability because they are subject to biases and flawed responses. For example, in some schools in other states, students who’ve been taught about issues like self-control rate themselves lower than their peers because they have a greater awareness of what those concepts mean.

Currently, “perfectly unbiased, unfakeable, and error-free measures are an ideal, not a reality,” researchers Angela Duckworth and David Yeager said in a May essay published in Educational Researcher that detailed an array of flaws with current measures.

Influential Stanford University psychology Professor Carol Dweck, who popularized the idea of growth mindset, said she doesn’t support the inclusion of that measure in the CORE districts’ system.

“To use the mindset measures directly for accountability is, I think, asking for problems,” Dweck said. Such practices may lead to shallow interventions that don’t actually improve student learning, she said, and they could indirectly encourage teachers “to be teaching their students how to check off the right box on a questionnaire.”

“I’m deeply pleased that they value the hard work that we’ve done, but then, that said, I have these concerns,” Dweck said.

Researchers with Transforming Education, a Boston-based organization that consulted with CORE on its social-emotional measures, expressed some concerns about bias in student responses, Bookman, the CORE accountability chief, said. But preliminary results show that strong student scores in noncognitive areas correlate with stronger academic performance and observable traits, like regularly turning in homework on time, which lends validity to the results, he added.

Sara Bartolino Krachman, the executive director of Transforming Education, said the questions used on student surveys passed several levels of pilot tests without showing signs of the biases researchers expected.

And CORE Executive Director Rick Miller said traits like self-efficacy and growth mindset are far too important for student success to delay measuring them.

Some of the same researchers who’ve criticized gauging such traits for accountability are actively working to develop more consistent, reliable measurements. Harvard Graduate School of Education Professor Martin West, for example, is working in Boston-area charter schools to devise new tools to measure student traits. Miller said CORE is open to changing its measures over time as new, more-sophisticated tools are introduced.

And, as a new federal accountability law leads to the end of the district-level waiver, the CORE districts intend to continue developing their model for comparing schools, whether or not it is tied to high-stakes federal accountability, he said.

Coverage of social and emotional learning is supported in part by a grant from the NoVo Foundation. Education Week retains sole editorial control over the content of this coverage.
Coverage of social and emotional learning is supported in part by a grant from the NoVo Foundation. Education Week retains sole editorial control over the content of this coverage.

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Teaching Webinar
Interactive Learning Best Practices: Creative Ways Interactive Displays Engage Students
Students and teachers alike struggle in our newly hybrid world where learning takes place partly on-site and partly online. Focus, engagement, and motivation have become big concerns in this transition. In this webinar, we will
Content provided by Samsung
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Classroom Technology Webinar
Educator-Driven EdTech Design: Help Shape the Future of Classroom Technology
Join us for a collaborative workshop where you will get a live demo of GoGuardian Teacher, including seamless new integrations with Google Classroom, and participate in an interactive design exercise building a feature based on
Content provided by GoGuardian
School & District Management Live Online Discussion A Seat at the Table With Education Week: What Did We Learn About Schooling Models This Year?
After a year of living with the pandemic, what schooling models might we turn to as we look ahead to improve the student learning experience? Could year-round schooling be one of them? What about online

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Student Well-Being Opinion Why Venting When You Have Problems Feels Good—and Why It Doesn’t Work
When you keep talking about what’s bothering you, it keeps the negative emotions alive. Here’s what research says to do instead.
Ethan Kross
2 min read
Images shows a stylized artistic landscape with soothing colors.
Getty
Student Well-Being What the Research Says How Does Sending a Child to School Change a Family's Risk of COVID-19?
In-person schooling that doesn't lead to outbreaks can still raise the risk of kids bringing the virus home, especially in poor families.
3 min read
On Sept. 24, 2020, distance learners are seen on a laptop held by teacher Kristen Giuliano who assists student Jane Wood, 11, in a seventh-grade social studies class at Dodd Middle School in Cheshire, Conn. A new study finds a family's risk of infection rose if they had a school-age student when schools re-started in person instruction.
Students, assisted by their teacher Kristen Giuliano, work remotely and in-person in a hybrid classroom earlier this year at Dodd Middle School in Cheshire, Conn.
Dave Zajac/Record-Journal via AP
Student Well-Being Teens Are Starting to Get Vaccinated. That's a Big Deal for Schools
Educators are now encouraging their oldest students to get the vaccine, with the hope that it will help normalize school operations.
10 min read
17-year-old cancer survivor Jordan Loughan receives a Pfizer vaccination at Mercedes-Benz Stadium in Atlanta on Tuesday, March 23, 2021.
Seventeen-year-old cancer survivor Jordan Loughan receives a Pfizer vaccination for COVID-19 in Atlanta on March 23.
Curtis Compton/Atlanta Journal-Constitution via AP
Student Well-Being Children as Young as 12 May Soon Be Able to Get Vaccinated
The Pfizer-BioNTech coronavirus vaccine is safe and effective for 12- to 15-year-olds, and that age group could be vaccinated before next school year.
6 min read
A clinical research nurse prepares to administer COVID-19 experimental vaccine to a volunteer at a clinic in London.
A clinical research nurse prepares to administer COVID-19 experimental vaccine to a volunteer at a clinic in London.
Kirsty Wigglesworth/AP-File