More Than a Snapshot
A major change is occurring in the way states collect education data. This change has two critical elements: state collection of student-level data, and the ability to match individual student records over time. These changes give states the ability to create longitudinal student data linking records of student enrollment, program participation, test scores, course completion, and graduation over multiple years. By contrast, traditional snapshot data provide only disconnected information on students at a moment in time.
|Matching student records longitudinally provides a rich store of data for evaluating the long-term impact of programs.|
Why is this change important for educators? Simply put, longitudinal data are far more useful for examining school and program effectiveness and identifying promising practices. For this reason, the shift to longitudinal student data should make a significant contribution to school improvement.
What can be learned from longitudinal student data? Using the data, educators, policymakers, and researchers can do the following:
- Follow students from high school to college, to find out what high school programs and courses are most strongly associated with college success. With information on students' level of academic preparation prior to entering high school, it is possible to learn what programs and courses work best with students who walk into the high school ill-prepared, average, or well- prepared.
- Identify the relationship between early achievement levels and later student success. This provides a basis for setting goals for school, district, or statewide early-intervention initiatives. For example, an analysis by our Texas-based nonprofit research organization, Just for the Kids, showed that students should reach the "proficiency" level on the state 8th grade mathematics exam to be well prepared for algebra in 9th grade. Further analysis showed that putting algebra off until 10th or 11th grade did little to raise scores, and that students who were not at least passing mathematics by 4th grade would probably not reach the required proficiency level by 8th grade. An analysis of longitudinal reading data found that only 27 percent of students who failed the state reading test in 3rd grade reached grade-level proficiency in reading by the end of 8th grade, and only 11 percent reached "mastery" levels in 8th grade writing. These analyses confirmed the importance of developing reading and mathematics skills in the early grades.
- Analyze the effects of specific state policies. Matching student records longitudinally provides a rich store of data for evaluating the long-term impact of early-childhood, bilingual, or dropout-prevention programs. For example, by following students from prekindergarten into 3rd grade, analysts at the Texas Education Agency were able to show the benefits of statewide investment in prekindergarten programs.
- Control for student mobility in reporting school test scores. Educators in high-mobility schools can benefit from knowing the success of students who have been exposed to their instruction for a significant period of time. Longitudinal data can address this issue by looking separately at the performance of students who have been enrolled in the same school for several years. If those students still haven't learned, that is likely to show the need for instructional changes at the school. Separate reporting on mobile and stable students can also direct attention to the special needs of mobile students.
- Improve the accuracy of socioeconomic data for high schools. Many students sign up for free or reduced-price lunches in middle school, but drop out of the program in high school. This makes the free-lunch data almost useless as a socioeconomic indicator for high schools. A better indicator would link student records back to middle school and count as low-income those high school students who participated in the free-lunch program in 7th or 8th grade.
- Create fair comparisons of middle and high schools. Having longitudinal data makes it possible to compare high schools' performance with students who walk in the door with similar levels of academic preparation. Comparing advanced-placement programs at two high schools, for example, may show that one school with an apparently lower success rate, but with students who are less well prepared at the beginning of 9th grade, may in fact have a higher success rate with comparably prepared students.
- Improve the investigation of promising practices. These investigations require identifying consistently high- and average-performing schools so that the practices in the two groups of schools may be compared. Longitudinal student data greatly improve our ability to identify which schools belong in these two categories—especially for middle and high schools, where students' prior academic preparation must be taken into account.
Using the data, educators, policymakers, and researchers can improve the accuracy of socioeconomic data for high schools.
The Education Commission of the States and Just for the Kids have begun an initiative to encourage states to collect longitudinal student data. In addition, the Parents' Right to Know Act, recently introduced in the U.S. Congress by Sen. Kay Bailey Hutchison of Texas encourages states to collect longitudinal data as a way to satisfy their Title I reporting requirements. Eleven states already have the ability to match student records. Several other states are upgrading their data-collection efforts to acquire this capability. Business and political leaders in those states have been active in developing a vision of how education will benefit from better information.
Why aren't more states developing the ability to create longitudinal student data? We believe there are several reasons:
- Political invisibility. Most states now produce school reports using snapshot data. On the surface, these reports look similar to charts or tables constructed with longitudinal data. This makes it easy to overlook the vastly superior usefulness and power of the latter. It's like comparing two identical-looking computer boxes, one containing 1979-vintage computer chips and software and the other packed with the very latest technology.
- Privacy. The creation of statewide longitudinal student data entails collecting and storing multiple years of student-level data at the state education agency and assigning each student a consistent statewide identification number that can be used to match records as students change schools and districts. Steps must be taken to safeguard student privacy. For example, when providing student-level data to researchers, states typically remove names, encrypt student IDs, and have researchers sign an agreement that they will not disclose results on groups of fewer than five students.
- Cost. State legislatures must appropriate the money required to create the necessary databases. Typically, the cost of doing so is less than one-tenth of 1 percent of the state education budget—a small price to pay for better information on how to spend the other 99.9 percent.
What can individuals do to promote the changeover from snapshot to longitudinal data?
|The Parents' Right to Know Act encourages states to collect longitudinal data as a way to satisfy their Title I reporting requirements.|
Educators, parents, and school board members can discuss the value of research using the longitudinal data available in their own schools, and the time and effort that educators can save if records can be matched by computer, instead of laboriously by hand.
State policymakers can lead efforts to encourage their own states to collect the necessary data and to make good use of the data that are already being collected.
And researchers can point out the benefits of studies that have relied on longitudinal data.
Expanding the use of longitudinal data is a critical step in helping education become a research-based, data-informed enterprise. This step is vital to the success of school improvement efforts.
Chrys Dougherty is the director of research of Just for the Kids , a nonprofit data and research organization based in Austin, Texas.
Vol. 20, Issue 33, Pages 39, 42Published in Print: May 2, 2001, as More Than a Snapshot