Includes updates and/or revisions.
While more states and districts are developing “early warning systems” to target students most at risk of dropping out, many of those systems may still not be reaching students early enough, according to the first national study to look at the data-based identification-and-intervention practice.
A study released Tuesday morning by Civic Enterprises, a Washington-based policy firm, and the Everyone Graduates Center at Johns Hopkins University in Baltimore found that 16 states now produce early warning systems that flag students who are not “on track” to graduate from high school, while 18 others have plans to implement such systems. Only four states so far—Delaware, New Jersey, Oklahoma and Virginia—provide early warning feedback to educators on a weekly or daily basis.
“A lot of these school districts and states are awash in data,” said John M. Bridgeland, the chief executive officer of Civic Enterprises and a co-author of the study. “The big problem has been using that data in a way that’s useful to teachers, administrators, and community-based nonprofits to target in on kids.”
“There are a lot of EWS’s (early warning systems) in schools across America, but a lot of them are put in place too late, in 9th grade,” Mr. Bridgeland said. “By the time kids are in 9th grade, a lot of them have already made their decisions about whether they are going to drop out.”
The researchers conducted detailed interviews and site visits at 16 “early adopter” districts in seven states. They found that many are already working on the next generation of indicator systems, intended to be more streamlined with information sent to teachers faster.
Varied Indicators
A growing number of states have early-warning data systems that flag students who are not “on track” to graduate from high school. Other states are in the process of developing such systems.
SOURCE: Data Quality Campaign
States and districts have experimented with both streamlined and complex systems of indicators—at one point, Louisiana’s system tracked and analyzed more than 200 separate indicators. At minimum, researchers found that most existing systems flag what Robert Balfanz, a co-author of the study and the director of the Everyone Graduates Center, calls “the ABCs” of such systems:
• Attendance: Students who have missed either 10 percent of school days or 20 days total;
• Behavior: Students who receive two or more mild or more-serious behavior citations, which in most schools means detentions or suspensions; and
• Course performance: Students who struggle to keep up in key classes at different grades.
The last indicator varies at critical transition grades. A student who can’t read on grade level by grade 3, when students begin moving beyond basic literacy to read to learn, is four times less likely to graduate by age 19 than a child who reads proficiently. Failing English or mathematics during grades 6 through 9 increases the chance a student will never catch up. The systems also tend to flag more general academic struggles associated with dropping out, such as a grade point average of less than 2.0, two or more failing grades in 9th grade, or not progressing from 9th to 10th grade on time.
While many state warning systems do include state assessment scores, Mr. Balfanz, whose Everyone Graduates Center has helped districts and states develop early warning systems, said school-based grades have proved more popular markers for educators. “Grades are a cumulative thing: Did you attend, did you try, did you get your work in? It includes all of those things,” Mr. Balfanz said. “The evidence shows [annual state] test scores are really not as predictive at the individual kid level.”
Flagging a student based on behavior can be problematic, the study found, because discipline policies vary so drastically from state to state and even school to school. Mr. Balfanz found that schools with “zero-tolerance” policies tend to have much higher overall discipline referrals, making it harder to tease out which students have the most severe underlying problems. He argued, however, that the number of discipline referrals will still predict students who are likelier to leave school, because “overreactive school policies” can prompt students to disengage.
Building Momentum
The momentum to develop and use these early warning systems has developed incredibly rapidly. While it usually takes a decade or two for interventions identified in research to translate into classroom practice, much of the research on dropout warning systems has come out in the last five years and researchers found that already 16 states have started producing regular early warning reports—and that’s not counting individual districts that have adopted their own systems.
Part of the impetus has come from increasing federal focus on increasing high school graduation. In the 2011-12 academic year, states, districts, and schools will for the first time be held accountable for their graduation rates based on a common federal metric in which cohorts of students entering 9th grade are tracked through graduation.
Yet the study shows that local businesses and community groups are also pushing districts to adopt early warning systems as part of grants or partnerships.
“There’s a sense of urgency in these local communities to get these things moving quickly,” Mr. Bridgeland said. Businesses and foundations in particular “were frustrated in grant-making [by] not getting sufficient feedback from school districts. They felt like they didn’t have the ability to understand the return on investment. ... [They] don’t want to wait for 10 years for a longitudinal study to show their investment is helping kids.”
The vast majority of the state and district early warning systems have been implemented only in the last year or two—not sufficient time to evaluate how well they are working overall—but the few that have been in place longer show promising results. For example, a review by the U.S. Department of Education’s What Works Clearinghouse found that the Check and Connect early warning program, which targets middle and high school students with learning, emotional, and behavioral disabilities, reduced truancy and helped students stay in school.
Moreover, while some states and districts adopted computerized data collection and analytics, which can produce detailed reports for teachers and researchers alike, the researchers found other districts simply gathered teachers and administrators together weekly or monthly to chart out students’ progress by hand on a bulletin board. “We were asking, ‘Does this have to be overwhelmingly costly for school systems?’ and there’s a strong case to be made that it’s really about more efficient use of existing resources,” Mr. Bridgeland said.
