Assessment

‘Nation’s Report Card’ to Gather Data on Grit, Mindset

By Sarah D. Sparks — June 02, 2015 | Corrected: February 21, 2019 4 min read

Corrected: In an earlier version of this story, a quote from ETS researcher Jan M. Alegre was incorrectly attributed. Chris Gabrieli’s work on developing new measures of noncognitive skills was incorrectly described. He is working with a group of seven California school districts on new measures.

The nation’s premiere federal testing program is poised to provide a critical window into how students’ motivation, mindset, and grit can affect their learning.

Evidence has been building for years that these so-called noncognitive factors play a role in whether children succeed both academically and socially. Now, the National Assessment of Educational Progress, often dubbed the “nation’s report card,” is working to include measures of these factors in the background information collected with the tests beginning in 2017.

“Teachers self-report spending 10 percent of their teaching time on noncognitive skills. That’s more time than students spend on any subject other than English and math—more than they spend on arts, for example,” said Chris Gabrieli, an adjunct lecturer with the Transforming Education project at Harvard Graduate School of Education and a co-founder of the National Center on Time & Learning in Boston.

“It’s not a question of whether schools are going to do more working on noncognitive factors,” he said, “it’s of whether we are going to have any instrumentation at all that lets us know which things are working and which things are not.”

Researchers from the Educational Testing Service described the project at a symposium here last month at the annual conference of the Association for Psychological Science. The background survey will include five core areas—grit, desire for learning, school climate, technology use, and socioeconomic status—of which the first two focus on a student’s noncognitive skills, and the third looks at noncognitive factors in the school. These core areas would be part of the background survey for all NAEP test-takers. In addition, questions about other noncognitive factors, such as self-efficacy and personal achievement goals, may be included on questionnaires for specific subjects to create content-area measures, according to Jonas P. Bertling, ETS director for NAEP survey questionnaires.

Careful Wording

Researchers tested different variations of the questions with 140 students in grades 4, 8, and 12 from a representative sample of ethnic and socioeconomic backgrounds in a three-state area around the District of Columbia. Students were led through a 90-minute interview, in which they answered the survey questions and then discussed their individual thought process in responding.

For example, students across all grades showed no difference in how they rated themselves on questions that asked them about their mindset in different contexts, and they reported preferring general questions rather than those specifically about school, said Jan M. Alegre, an ETS researcher.

“A majority of students preferred questions that went beyond a simple yes or no, whether they did something or not,” she said.

Small changes in phrasing make a difference in how well students respond to the questions. For example, “4th graders didn’t know what ‘thinking abstractly’ meant. ... Students had difficulty describing what experiencing failure meant and what it meant to be committed,” Ms. Alegre said. As a result, the researchers changed survey questions asking whether students had “experienced failure” to “making mistakes” and changed “committed to goals” to “continue to work toward my goals.”

The background questions will go through a third and final round of review in spring 2016, before the questions are administered beginning with tests in 2017.

Not for Accountability

Schools will not be judged based on the NAEP noncognitive measures of their students, but other such tests for accountability purposes may be on the horizon.

A coalition of seven California districts that have received waivers from some federal accountability requirements are developing a new accountability system, in which 40 percent of a school’s evaluation will take into account school culture and students’ social and emotional learning. Within the latter section, researchers are completing the field testing of growth mindset, self-efficacy and self-management, and social awareness measures with 9,000 students and 1,000 teachers. Mr. Gabrieli said the new measures are expected to be in place next year. He is helping to develop the new measures.

However, while poor performance on academic accountability measures can lead to sanctions in many districts, coalition schools with poor ratings for noncognitive skills will simply be paired with a higher-scoring mentor school.

As measures of noncognitive skills become more ubiquitous, Mr. Gabrieli said, it will be important to track disparities between students’ reports of their mindset and tenacity, and teachers’ observations of them.

“I have often seen in data collected in smaller samples this tendency for teachers to rate students on separate [questions] basically the same, as if they have one view of Johnny as a good kid or less-good kid,” Mr. Gabrieli said. “It’s hard to get teachers to follow the rules of ‘this construct is about self-regulation and that one is about interpersonal skills.’ ”

A version of this article appeared in the June 03, 2015 edition of Education Week as NAEP to Gather Data on Grit, Mindset

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Classroom Technology Webinar
Educator-Driven EdTech Design: Help Shape the Future of Classroom Technology
Join us for a collaborative workshop where you will get a live demo of GoGuardian Teacher, including seamless new integrations with Google Classroom, and participate in an interactive design exercise building a feature based on
Content provided by GoGuardian
School & District Management Live Online Discussion A Seat at the Table With Education Week: What Did We Learn About Schooling Models This Year?
After a year of living with the pandemic, what schooling models might we turn to as we look ahead to improve the student learning experience? Could year-round schooling be one of them? What about online
School & District Management Webinar What's Ahead for Hybrid Learning: Putting Best Practices in Motion
It’s safe to say hybrid learning—a mix of in-person and remote instruction that evolved quickly during the pandemic—is probably here to stay in K-12 education to some extent. That is the case even though increasing

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment Opinion Alternatives to Standardized Tests During a Pandemic Year
Three educators suggest alternatives to federally mandated standardized testing during this year undercut by COVID-19.
7 min read
Images shows colorful speech bubbles that say "Q," "&," and "A."
iStock/Getty
Assessment Opinion AP Exams Can't Be Business as Usual This Year
The College Board seems unconcerned with the collateral damage of its pandemic approach, writes an assistant superintendent of curriculum and instruction.
Pete Bavis
5 min read
Illustration of large boat in turbulent waters with other smaller boats falling into the abyss.
iStock/Getty Images Plus
Assessment Federal Lawmakers Urge Miguel Cardona to Let States Cancel Tests, Highlighting Discord
A letter from Democratic members to the new education secretary calls for an end to the "flawed" system of annual standardized exams.
3 min read
Jamaal Bowman speaks to reporters after voting at a polling station inside Yonkers Middle/High School in Yonkers, N.Y. on June 23, 2020.
Jamaal Bowman speaks to reporters after voting at a polling station inside Yonkers Middle/High School in Yonkers, N.Y. on June 23, 2020.
John Minchillo/AP
Assessment How Two Years of Pandemic Disruption Could Shake Up the Debate Over Standardized Testing
Moves to opt out of state tests and change how they're given threaten to reignite fights over high-stakes assessments.
9 min read
Image of a student at a desk.
patat/iStock/Getty