Accountability

States Need Updates for Managing Data, Analysis Concludes

By David J. Hoff — October 22, 2003 6 min read
  • Save to favorites
  • Print

States will need to make major upgrades to their data-collection systems in order to take full advantage of the mounting wealth of student test results, according to a new analysis from a group tracking state accountability plans.

“The Value of Longitudinal Student Data” is available from the National Center for Educational Accountability.

Just 21 states, for example, are tracking individual student performance with a so-called student identifier that records achievement across a pupil’s K-12 career—a key element in interpreting test scores and using them to improve school policies, according to the National Center for Educational Accountability.

Without such information, state officials aren’t able to figure out what they can do to raise achievement for struggling students, or for others who have similar demographic characteristics, said Chrys Dougherty, the research director for the Austin, Texas, nonprofit group.

“You just can’t interpret [a student’s test scores] unless you know how they were when they came into the school,” Mr. Dougherty said in an interview last week. “It’s absolutely essential if you’re going to do any kind of analysis at the middle and high school level.”

In fact, unless a state has a way to track achievement, student by student, it will struggle to comply with the accountability mandates in the federal No Child Left Behind Act.

“It’s implicitly called for in No Child Left Behind,” said Margaret E. Raymond, the director of education policy and evaluation at the Hoover Institution, an independent think tank at Stanford University. She joined the NCEA advisory board after the study was completed this fall. “It’s the next innovation in student-information systems.”

States are working hard to upgrade data collection, according to Mr. Dougherty and Ms. Raymond. But overall, the two analysts said, states lack student identifiers and many basic ingredients that Mr. Dougherty’s group suggests they should have to get the full value from their test scores.

IDs and More

After surveying state officials and researchers, the National Center for Educational Accountability identified the nine “essential elements” of state data systems.

Using Data
9 Key Elements of Statewide Data-Collection Systems

1. Unique statewide student identifier.
2. Fall student-level enrollment data.
3. Student-level test data.
4. Information on untested students.
5. Student-level course-completion data.
6. Student-level SAT, ACT, and AP test results.
7. Student-level graduation and dropout rates.
8. Ability to match K-12 and higher education.
9. State data-audit system.

SOURCE: National Center for Educational Accountability

First on the list is a unique code that identifies each student so that the state can follow enrollment, test scores, and other factors as the student matriculates. The identifiers also help track students as they migrate from school to school within districts or across a state.

Other elements on the list include collecting information on individual student coursetaking, recording scores on SAT, ACT, and Advanced Placement tests, and following student success in higher education. The final piece is to ensure the state has a thorough audit of all the data it collects and receives from districts.

So far, only Texas and Florida have all the pieces of what a NCEA brochure calls the “Nine Essential Elements of Statewide Data-Collection Systems.” Georgia and Louisiana have eight of the nine elements.

But 22 states have three or fewer of the pieces, according to the NCEA’s state-by-state analysis.

States are making progress, however. Since 1999, the NCEA found, 13 states have added student identifiers to their data systems, and others are working to add them.

“The progress is very strong in some states, and very tentative or limited in other states,” Mr. Dougherty said.

States may soon receive federal subsidies to upgrade their databases. The U.S. Senate’s fiscal 2004 appropriations bill would institute an $80 million competitive-grant program to help states with efforts to create data systems needed to comply with the No Child Left Behind Act.

House and Senate negotiators are now working out a compromise on their differing appropriations bills.

States have to be careful in choosing what information they want, one researcher said.

“In the past, people have created real beasts,” said Joan L. Herman, a co-director of the National Center for Research on Evaluation, Standards, and Student Testing, based at the University of California, Los Angeles. “You run the risk of using time and energy collecting data that no one ever uses.”

Keeping Track

The need for student identifiers—and other individual student data—reflects the movement in the school reform debate from tracking schools’ or districts’ progress toward learning goals, to following individual students’ achievement of those goals.

Just for the Kids Inc.—the National Center for Educational Accountability’s parent organization—has used school- level information collected under the Texas testing program to publish detailed analyses of Texas schools’ performance on state tests. The Education Commission of the States and the University of Texas at Austin are the Austin- based Just for the Kids’ partners in the NCEA.

In addition to using data in new ways, states will soon be collecting a huge amount of fresh statistics when complying with the No Child Left Behind Act’s requirement that states assess the reading abilities and math skills of every student from the 3rd through the 8th grade, and once in high school.

All that information will do little good, though, if it isn’t used to make a difference, according to Matt Gandal, the executive vice president of Achieve, a Washington-based group led by governors and corporate executives that promotes increased student achievement.

“This stuff has to come alive and be used in schools and statehouses to make important decisions,” Mr. Gandal said.

For example, Ohio policymakers are starting to reap the benefits of creating a unique code identifying every student. Before, the state could analyze test scores only at the district level, according to Kimberly A. Murnieks, the deputy chief of staff for the Ohio Department of Education.

Now, in addition to providing statistics that help schools make informed decisions about individuals, the student information “sets us up nicely” to comply with the accountability and reporting requirements of the No Child Left Behind Act, Ms. Murnieks said.

Auditing Role

In addition to collecting and reporting data, states should conduct thorough audits of that information, the NCEA suggests.

Such audits should include statistical reviews of district data to flag anything that looks suspicious, as well as random inspections of the data a district collects, the center says.

The recent firestorm in Houston over misreported dropout figures demonstrates the importance of auditing data to ensure the information reflects what’s actually happening in schools, Mr. Dougherty said. (“Houston Case Offers Lesson on Dropouts,” Sept. 24, 2003.)

Ohio’s new individual student-level data give the state another source of information to double-check when it audits the statistics from districts on dropouts and other issues, Ms. Murnieks added.

Events

Jobs Virtual Career Fair for Teachers and K-12 Staff
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.
Ed-Tech Policy Webinar Artificial Intelligence in Practice: Building a Roadmap for AI Use in Schools
AI in education: game-changer or classroom chaos? Join our webinar & learn how to navigate this evolving tech responsibly.
Education Webinar Developing and Executing Impactful Research Campaigns to Fuel Your Ed Marketing Strategy 
Develop impactful research campaigns to fuel your marketing. Join the EdWeek Research Center for a webinar with actionable take-aways for companies who sell to K-12 districts.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Accountability Opinion What’s Wrong With Online Credit Recovery? This Teacher Will Tell You
The “whatever it takes” approach to increasing graduation rates ends up deflating the value of a diploma.
5 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
Accountability Why a Judge Stopped Texas from Issuing A-F School Ratings
Districts argued the new metric would make it appear as if schools have worsened—even though outcomes have actually improved in many cases.
2 min read
Laura BakerEducation Week via Canva  (1)
Canva
Accountability Why These Districts Are Suing to Stop Release of A-F School Ratings
A change in how schools will be graded has prompted legal action from about a dozen school districts in Texas.
4 min read
Handwritten red letter grades cover a blue illustration of a classic brick school building.
Laura Baker, Canva
Accountability What the Research Says What Should Schools Do to Build on 20 Years of NCLB Data?
The education law yielded a cornucopia of student information, but not scalable turnaround for schools, an analysis finds.
3 min read
Photo of magnifying glass and charts.
iStock / Getty Images Plus