States Need Updates for Managing Data, Analysis Concludes

By David J. Hoff — October 22, 2003 6 min read
  • Save to favorites
  • Print

States will need to make major upgrades to their data-collection systems in order to take full advantage of the mounting wealth of student test results, according to a new analysis from a group tracking state accountability plans.

“The Value of Longitudinal Student Data” is available from the National Center for Educational Accountability.

Just 21 states, for example, are tracking individual student performance with a so-called student identifier that records achievement across a pupil’s K-12 career—a key element in interpreting test scores and using them to improve school policies, according to the National Center for Educational Accountability.

Without such information, state officials aren’t able to figure out what they can do to raise achievement for struggling students, or for others who have similar demographic characteristics, said Chrys Dougherty, the research director for the Austin, Texas, nonprofit group.

“You just can’t interpret [a student’s test scores] unless you know how they were when they came into the school,” Mr. Dougherty said in an interview last week. “It’s absolutely essential if you’re going to do any kind of analysis at the middle and high school level.”

In fact, unless a state has a way to track achievement, student by student, it will struggle to comply with the accountability mandates in the federal No Child Left Behind Act.

“It’s implicitly called for in No Child Left Behind,” said Margaret E. Raymond, the director of education policy and evaluation at the Hoover Institution, an independent think tank at Stanford University. She joined the NCEA advisory board after the study was completed this fall. “It’s the next innovation in student-information systems.”

States are working hard to upgrade data collection, according to Mr. Dougherty and Ms. Raymond. But overall, the two analysts said, states lack student identifiers and many basic ingredients that Mr. Dougherty’s group suggests they should have to get the full value from their test scores.

IDs and More

After surveying state officials and researchers, the National Center for Educational Accountability identified the nine “essential elements” of state data systems.

Using Data
9 Key Elements of Statewide Data-Collection Systems

1. Unique statewide student identifier.
2. Fall student-level enrollment data.
3. Student-level test data.
4. Information on untested students.
5. Student-level course-completion data.
6. Student-level SAT, ACT, and AP test results.
7. Student-level graduation and dropout rates.
8. Ability to match K-12 and higher education.
9. State data-audit system.

SOURCE: National Center for Educational Accountability

First on the list is a unique code that identifies each student so that the state can follow enrollment, test scores, and other factors as the student matriculates. The identifiers also help track students as they migrate from school to school within districts or across a state.

Other elements on the list include collecting information on individual student coursetaking, recording scores on SAT, ACT, and Advanced Placement tests, and following student success in higher education. The final piece is to ensure the state has a thorough audit of all the data it collects and receives from districts.

So far, only Texas and Florida have all the pieces of what a NCEA brochure calls the “Nine Essential Elements of Statewide Data-Collection Systems.” Georgia and Louisiana have eight of the nine elements.

But 22 states have three or fewer of the pieces, according to the NCEA’s state-by-state analysis.

States are making progress, however. Since 1999, the NCEA found, 13 states have added student identifiers to their data systems, and others are working to add them.

“The progress is very strong in some states, and very tentative or limited in other states,” Mr. Dougherty said.

States may soon receive federal subsidies to upgrade their databases. The U.S. Senate’s fiscal 2004 appropriations bill would institute an $80 million competitive-grant program to help states with efforts to create data systems needed to comply with the No Child Left Behind Act.

House and Senate negotiators are now working out a compromise on their differing appropriations bills.

States have to be careful in choosing what information they want, one researcher said.

“In the past, people have created real beasts,” said Joan L. Herman, a co-director of the National Center for Research on Evaluation, Standards, and Student Testing, based at the University of California, Los Angeles. “You run the risk of using time and energy collecting data that no one ever uses.”

Keeping Track

The need for student identifiers—and other individual student data—reflects the movement in the school reform debate from tracking schools’ or districts’ progress toward learning goals, to following individual students’ achievement of those goals.

Just for the Kids Inc.—the National Center for Educational Accountability’s parent organization—has used school- level information collected under the Texas testing program to publish detailed analyses of Texas schools’ performance on state tests. The Education Commission of the States and the University of Texas at Austin are the Austin- based Just for the Kids’ partners in the NCEA.

In addition to using data in new ways, states will soon be collecting a huge amount of fresh statistics when complying with the No Child Left Behind Act’s requirement that states assess the reading abilities and math skills of every student from the 3rd through the 8th grade, and once in high school.

All that information will do little good, though, if it isn’t used to make a difference, according to Matt Gandal, the executive vice president of Achieve, a Washington-based group led by governors and corporate executives that promotes increased student achievement.

“This stuff has to come alive and be used in schools and statehouses to make important decisions,” Mr. Gandal said.

For example, Ohio policymakers are starting to reap the benefits of creating a unique code identifying every student. Before, the state could analyze test scores only at the district level, according to Kimberly A. Murnieks, the deputy chief of staff for the Ohio Department of Education.

Now, in addition to providing statistics that help schools make informed decisions about individuals, the student information “sets us up nicely” to comply with the accountability and reporting requirements of the No Child Left Behind Act, Ms. Murnieks said.

Auditing Role

In addition to collecting and reporting data, states should conduct thorough audits of that information, the NCEA suggests.

Such audits should include statistical reviews of district data to flag anything that looks suspicious, as well as random inspections of the data a district collects, the center says.

The recent firestorm in Houston over misreported dropout figures demonstrates the importance of auditing data to ensure the information reflects what’s actually happening in schools, Mr. Dougherty said. (“Houston Case Offers Lesson on Dropouts,” Sept. 24, 2003.)

Ohio’s new individual student-level data give the state another source of information to double-check when it audits the statistics from districts on dropouts and other issues, Ms. Murnieks added.


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Equity & Diversity Webinar
Classroom Strategies for Building Equity and Student Confidence
Shape equity, confidence, and success for your middle school students. Join the discussion and Q&A for proven strategies.
Content provided by Project Lead The Way
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Professional Development Webinar
Disrupting PD Day in Schools with Continuous Professional Learning Experiences
Hear how this NC School District achieved district-wide change by shifting from traditional PD days to year-long professional learning cycles
Content provided by BetterLesson
Jobs Virtual Career Fair for Teachers and K-12 Staff
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Accountability Timeline: How Federal School Accountability Has Waxed and Waned
From its origins in the 1990s to the most-recent tack, see how the federal approach to accountability has shifted.
4 min read
President George W. Bush, left, participates in the swearing-in ceremony for the Secretary of Education Margaret Spellings, center, at the U.S. Dept. of Education on Jan. 31, 2005 in Washington. On the far right holding a bible is her husband Robert Spellings.
President George W. Bush, left, participates in the swearing-in ceremony for the Secretary of Education Margaret Spellings, center, at the U.S. Dept. of Education on Jan. 31, 2005 in Washington. On the far right holding a bible is her husband Robert Spellings.
AP Photo/Pablo Martinez Monsivais
Accountability School Accountability Is Restarting After a Two-Year Pause. Here's What That Means
For a moment, the COVID-19 pandemic succeeded in doing what periodic protests about school accountability couldn't: Halting it.
10 min read
Illustration of a gauge.
Accountability Opinion Let's Take a Holistic Approach to Judging Schools
Parents wouldn't judge their kids based on a single factor. So, says Ron Berger of EL Education, why must schools use a lone test score?
8 min read
Images shows colorful speech bubbles that say "Q," "&," and "A."
Accountability Opinion Are K-12 State Tests Like a Visit to the Pediatrician?
Even if the doctor’s trip isn’t pleasant, at least parents get something out of it they believe is worthwhile.
3 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty