Every Student Succeeds Act

Data Looms Large in Quest for New School-Quality Indicator

By Daarel Burnette II — July 19, 2016 6 min read
  • Save to favorites
  • Print

States scrambling to come up with more nuanced ways to measure school quality under the new federal K-12 law are running smack into an old problem: how to make sure they have the right data.

The Every Student Succeeds Act requires that states—in addition to using English-language proficiency, graduation rates, and scores on statewide achievement tests—add at least one new indicator of school quality or student success, such as school climate, chronic absenteeism, discipline, or college and career readiness.

For many states, adding that new indicator may mean spending more on data systems and collection, avoiding approaches that might demand too much of a data lift, or picking something off the shelf rather than crafting a more challenging indicator, because the information isn’t easily available.

Complicating the matter, the law requires that the data for the new school-quality indicator must be valid, reliable, and comparable across districts, and that officials be able to break out the information by student demographics.

That presents a challenge for state education agencies that want to pick indicators that use classroom observations or teacher and parent surveys to measure schoolwide indicators. Those might include whether parents feel engaged or if teachers are participating in effective peer-mentor programs, for example.

“Here’s a great opportunity for departments to innovate, and they’re being placed right back in a box,” said Mark Elgart, the president and chief executive officer of AdvancEd, a group that’s consulted with education departments to help them create new accountability systems.

But many consultants working with state departments are advising that they not let data-collection issues impede innovation.

“If something’s not feasible to collect, you have to treat it as an implementation issue,” said Joanne Weiss, who was a chief of staff to former U.S. Secretary of Education Arne Duncan and who currently consults with state education departments. “That doesn’t mean it’s not an important indicator that shouldn’t be included in the system.”

Tough Choices

The U.S. Department of Education in June issued its proposed regulations for states putting together new accountability systems under ESSA, which is due to go fully into effect in fall 2017. The draft repeats the law’s requirements for four mandated academic indicators, as well as for the new “fifth indicator” of school quality or student success.

Heavy Lift

As states revamp their accountability systems under the Every Student Succeeds Act, some are wrestling with what it will take to collect the data needed to incorporate new indicators into those systems.

CALIFORNIA
Indicators: Considering indicators in four areas: chronic absenteeism; suspension rates; college and career readiness; school climate.
Time Frame: Data collection starts in September; system rolls out fall of 2017.
Data Specifics: Work needed to collect clear, consistent, up-to-date data across districts in certain areas.

CONNECTICUT
Indicators: Adopted five indicators: attendance/chronic absence; college and career readiness; postsecondary entrance; physical fitness; arts access.
Time Frame: The state plans to submit new accountability system to U.S. Department of Education after making minor tweaks to comply with the recently released regulations.
Data Specifics: State for several years has collected all the data required to measure the indicators picked for the accountability system.

SOUTH CAROLINA
Indicators: Refining indicators in four areas: elementary school readiness; middle school readiness; high school readiness; college and career readiness.
Time Frame: End of summer to refine indicators, accountability system to roll out at the start of the 2017-18 school year.
Data Specifics: New, centralized data-management system estimated to cost more than $1 million. The system will flag course work schools offer and how students perform in those courses.

Sources: California, Connecticut, South Carolina Departments of Education

In the meantime, many states already are wrestling with whether to pick a school quality indicator that is ideal and ambitious, versus one that is practical and safe, with data collection and analysis a major factor.

• California’s education department has pushed back against aggressive efforts by parent and advocates to measure school climate, an indicator that officials say they don’t yet have enough reliable information to measure.

• Connecticut’s education department rejected proposals to add civic engagement to that state’s accountability system—an indicator that would require collecting new data.

• And South Carolina officials, not wanting to trample on the state’s accountability task force’s imagination, will spend more than $1 million to measure school and career readiness as part of its new accountability system. “We know we’re going to be collecting significantly more data with this new system,” said Sheila Quinn, South Carolina’s deputy schools superintendent.

One big issue: whether states and districts are able to retrofit their data-collection systems to answer new and increasingly difficult questions, a potentially arduous and expensive task.

For many measures, state officials say they lack the infrastructure to collect enough reliable information to attach high stakes. Many districts’ data-collection sytems are scattershot and outdated. Scores of technicians responsible for processing data have been laid off in recent years amid budget cuts. And local superintendents have complained that they’re already required by states to collect an inordinate amount of data.

The details are daunting. Scott Norton, the Council of Chief State School Officers’ strategic-initiative director for standards, assessment, and accountability, said pulling all the right data together requires syncing districts’ systems, then coding those systems to collect the right information.

Some data points, such as whether a student is a foster child or part of a military family, are pretty straightforward. But others—such as how students feel about a school’s climate or whether teachers are receiving a certain amount of professional development—may require a bevy of surveys that then must be manually entered into the database.

As a result, many education departments, depending on their capacity, will consider outsourcing the work or paying millions of dollars to purchase entire new systems, consultants say.

Student-Level Information

There’s also the sheer volume of information. School districts today collect hundreds of thousands of data points about children that are often stored in large data warehouses. Students track their academic progress in data binders, teachers tweak their curriculum based on rapid-fire online quizzes, and principals tally office referrals to craft new discipline procedures.

Against that backdrop, Brennan McMahon Parton, the Data Quality Campaign’s associate director for state policy and advocacy, has traversed the country in recent months urging state education departments and lawmakers to evaluate data they already collect before deciding to collect more as they weigh new school quality indicators.

“Many states have meaningful and useful data in their system already,” she said. “That’s not to say with a push of a button, you get what you need.”

In Connecticut, more than two-thirds of local superintendents said in a 2012 survey that the amount of data the state requires them to collect was duplicative, burdensome, and costly. That year, Democratic Gov. Dannel Malloy signed an education bill that tasked the state department to reduce by a third the number of data forms districts annually fill out.

So when the education department formed a task force two years ago to construct a new accountability system, superintendents and the agency pledged that any new indicators would have to be based on information the department already collected.

“Oftentimes, when the state asks for new data, we tell them we already have it or we’ve been giving it to you in other ways,” said Joseph J. Cirasuolo, the executive director of Connecticut’s superintendents association. “Usually, it’s not where it has to be.”

In the end, Connecticut decided to add access to arts courses, chronic absenteeism, career readiness (based on students’ performance on the state’s achievement test, SAT, ACT, Advanced Placement, or International Baccalaureate tests), schools’ college-entrance rates, and three new ways to measure graduation rates to its accountability system.

Big Price Tag

In South Carolina, the task force designated to come up with a new accountability system decided to collect information on elementary, middle and high school readiness and career readiness. That state’s districts all collect data using separate systems, many with different contractors. Definitions of indicators such as chronic absenteeism or what qualifies as a suspension vary widely.

In order for South Carolina to measure the new indicators, the department will spend more than $1 million to buy a new collection system that pulls data points from each district’s systems.

“We collect attendance, but the question is: What is the quality of the attendance data that we receive?” asked Daniel Ralyea, the director of the state education department’s office of research and data analysis. “I can aggregate it at the state level, but what happens is, in practice, elementary schools may not be as concerned with recording attendance as high schools are.”

And in California, the debate over whether to use school climate as an indicator involves such factors as classroom observations and a host of student, parent, and teacher surveys.

“We think (those surveys) are used best at the local level,” said Keric Ashley, the deputy superintendent of California’s education department, pointing out that the data are prone to errors.

A version of this article appeared in the July 20, 2016 edition of Education Week as Data Loom Large in Quest for New School-Quality Indicator

Events

Jobs Virtual Career Fair for Teachers and K-12 Staff
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.
Ed-Tech Policy Webinar Artificial Intelligence in Practice: Building a Roadmap for AI Use in Schools
AI in education: game-changer or classroom chaos? Join our webinar & learn how to navigate this evolving tech responsibly.
Education Webinar Developing and Executing Impactful Research Campaigns to Fuel Your Ed Marketing Strategy 
Develop impactful research campaigns to fuel your marketing. Join the EdWeek Research Center for a webinar with actionable take-aways for companies who sell to K-12 districts.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Every Student Succeeds Act Opinion 20 Years Ago, NCLB Kinda, Sorta Worked. That's the Problem
NCLB's political success gave rise to a more complicated reality of lax academic standards and public cynicism.
3 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
Every Student Succeeds Act Biden Education Department Approves One Request to Cancel State Tests But Rejects Others
Officials will allow D.C. to cancel tests. They denied similar requests from two other states and approved less extensive waiver requests.
6 min read
Image of students taking a test.
smolaw11/iStock/Getty
Every Student Succeeds Act Republicans Tell Miguel Cardona His Plan for ESSA Waivers Seems to Violate the Law
The Every Student Succeeds Act doesn't permit the education secretary to seek certain data he's asking for, the two GOP lawmakers say.
4 min read
White House press secretary Jen Psaki, left, listens as Education Secretary Miguel Cardona, center, speaks during a press briefing at the White House on March 17, 2021.
White House press secretary Jen Psaki, left, listens as Education Secretary Miguel Cardona, center, speaks during a press briefing at the White House on March 17, 2021.
Andrew Harnik/AP
Every Student Succeeds Act How Will ESSA Hold Up During COVID-19? Pandemic Tests the Law's Resilience
Lawmakers designed ESSA to limit mandates covering issues like how tests are used. Will that affect how well the law survives the pandemic?
6 min read