States Need Updates for Managing Data, Analysis Concludes

By David J. Hoff — October 22, 2003 6 min read

States will need to make major upgrades to their data-collection systems in order to take full advantage of the mounting wealth of student test results, according to a new analysis from a group tracking state accountability plans.

“The Value of Longitudinal Student Data” is available from the National Center for Educational Accountability.

Just 21 states, for example, are tracking individual student performance with a so-called student identifier that records achievement across a pupil’s K-12 career—a key element in interpreting test scores and using them to improve school policies, according to the National Center for Educational Accountability.

Without such information, state officials aren’t able to figure out what they can do to raise achievement for struggling students, or for others who have similar demographic characteristics, said Chrys Dougherty, the research director for the Austin, Texas, nonprofit group.

“You just can’t interpret [a student’s test scores] unless you know how they were when they came into the school,” Mr. Dougherty said in an interview last week. “It’s absolutely essential if you’re going to do any kind of analysis at the middle and high school level.”

In fact, unless a state has a way to track achievement, student by student, it will struggle to comply with the accountability mandates in the federal No Child Left Behind Act.

“It’s implicitly called for in No Child Left Behind,” said Margaret E. Raymond, the director of education policy and evaluation at the Hoover Institution, an independent think tank at Stanford University. She joined the NCEA advisory board after the study was completed this fall. “It’s the next innovation in student-information systems.”

States are working hard to upgrade data collection, according to Mr. Dougherty and Ms. Raymond. But overall, the two analysts said, states lack student identifiers and many basic ingredients that Mr. Dougherty’s group suggests they should have to get the full value from their test scores.

IDs and More

After surveying state officials and researchers, the National Center for Educational Accountability identified the nine “essential elements” of state data systems.

Using Data
9 Key Elements of Statewide Data-Collection Systems

1. Unique statewide student identifier.
2. Fall student-level enrollment data.
3. Student-level test data.
4. Information on untested students.
5. Student-level course-completion data.
6. Student-level SAT, ACT, and AP test results.
7. Student-level graduation and dropout rates.
8. Ability to match K-12 and higher education.
9. State data-audit system.

SOURCE: National Center for Educational Accountability

First on the list is a unique code that identifies each student so that the state can follow enrollment, test scores, and other factors as the student matriculates. The identifiers also help track students as they migrate from school to school within districts or across a state.

Other elements on the list include collecting information on individual student coursetaking, recording scores on SAT, ACT, and Advanced Placement tests, and following student success in higher education. The final piece is to ensure the state has a thorough audit of all the data it collects and receives from districts.

So far, only Texas and Florida have all the pieces of what a NCEA brochure calls the “Nine Essential Elements of Statewide Data-Collection Systems.” Georgia and Louisiana have eight of the nine elements.

But 22 states have three or fewer of the pieces, according to the NCEA’s state-by-state analysis.

States are making progress, however. Since 1999, the NCEA found, 13 states have added student identifiers to their data systems, and others are working to add them.

“The progress is very strong in some states, and very tentative or limited in other states,” Mr. Dougherty said.

States may soon receive federal subsidies to upgrade their databases. The U.S. Senate’s fiscal 2004 appropriations bill would institute an $80 million competitive-grant program to help states with efforts to create data systems needed to comply with the No Child Left Behind Act.

House and Senate negotiators are now working out a compromise on their differing appropriations bills.

States have to be careful in choosing what information they want, one researcher said.

“In the past, people have created real beasts,” said Joan L. Herman, a co-director of the National Center for Research on Evaluation, Standards, and Student Testing, based at the University of California, Los Angeles. “You run the risk of using time and energy collecting data that no one ever uses.”

Keeping Track

The need for student identifiers—and other individual student data—reflects the movement in the school reform debate from tracking schools’ or districts’ progress toward learning goals, to following individual students’ achievement of those goals.

Just for the Kids Inc.—the National Center for Educational Accountability’s parent organization—has used school- level information collected under the Texas testing program to publish detailed analyses of Texas schools’ performance on state tests. The Education Commission of the States and the University of Texas at Austin are the Austin- based Just for the Kids’ partners in the NCEA.

In addition to using data in new ways, states will soon be collecting a huge amount of fresh statistics when complying with the No Child Left Behind Act’s requirement that states assess the reading abilities and math skills of every student from the 3rd through the 8th grade, and once in high school.

All that information will do little good, though, if it isn’t used to make a difference, according to Matt Gandal, the executive vice president of Achieve, a Washington-based group led by governors and corporate executives that promotes increased student achievement.

“This stuff has to come alive and be used in schools and statehouses to make important decisions,” Mr. Gandal said.

For example, Ohio policymakers are starting to reap the benefits of creating a unique code identifying every student. Before, the state could analyze test scores only at the district level, according to Kimberly A. Murnieks, the deputy chief of staff for the Ohio Department of Education.

Now, in addition to providing statistics that help schools make informed decisions about individuals, the student information “sets us up nicely” to comply with the accountability and reporting requirements of the No Child Left Behind Act, Ms. Murnieks said.

Auditing Role

In addition to collecting and reporting data, states should conduct thorough audits of that information, the NCEA suggests.

Such audits should include statistical reviews of district data to flag anything that looks suspicious, as well as random inspections of the data a district collects, the center says.

The recent firestorm in Houston over misreported dropout figures demonstrates the importance of auditing data to ensure the information reflects what’s actually happening in schools, Mr. Dougherty said. (“Houston Case Offers Lesson on Dropouts,” Sept. 24, 2003.)

Ohio’s new individual student-level data give the state another source of information to double-check when it audits the statistics from districts on dropouts and other issues, Ms. Murnieks added.


School & District Management Live Online Discussion A Seat at the Table: Strategies & Tips for Complex Decision-Making
Schools are working through the most disruptive period in the history of modern education, facing a pandemic, economic problems, social justice issues, and rapid technological change all at once. But even after the pandemic ends,
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Education Funding Webinar
From Crisis to Opportunity: How Districts Rebuild to Improve Student Well-Being
K-12 leaders discuss the impact of federal funding, prioritizing holistic student support, and how technology can help.
Content provided by
Classroom Technology Online Summit Technology & the Pandemic: What’s Next for Schools?
When it comes to the use of technology, what’s next for schools?  Join the discussion to tackle issues surrounding this important question.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Accountability Biden Education Team Squashes States' Push to Nix All Tests but Approves Other Flexibility
The department has telegraphed its decision to deny states' requests to cancel federally mandated tests for weeks.
3 min read
A first-grader learns keyboarding skills at Bayview Elementary School in San Pablo, Calif on March 12, 2015. Schools around the country are teaching students as young as 6 years old, basic typing and other keyboarding skills. The Common Core education standards adopted by a majority of states call for students to be able to use technology to research, write and give oral presentations, but the imperative for educators arrived with the introduction of standardized tests that are taken on computers instead of with paper and pencils.
The U.S. Department of Education denied some states' requests to cancel standardized tests this year. Others are seeking flexibility from some testing requirements, rather than skipping the assessments altogether.
Eric Risberg/AP
Accountability Explainer Will There Be Standardized Tests This Year? 8 Questions Answered
Educators want to know: Will the exams happen? If so, what will they look like, and how will the results be used?
12 min read
Students testing.
Accountability Opinion What Should School Accountability Look Like in a Time of COVID-19?
Remote learning is not like in person, and after nine months of it, data are revealing how harmful COVID-19 has been to children's learning.
6 min read
Image shows a speech bubble divided into 4 overlapping, connecting parts.
DigitalVision Vectors/Getty and Laura Baker/Education Week
Accountability State Schools Chiefs Push Biden for Wiggle Room on Accountability During Pandemic
State schools chiefs say it's necessary to change how they use scores from mandated annual tests during the unprecedented disruption created by the coronavirus pandemic.
4 min read
Image of students taking a test.