The U.S. Department of Education may not have a good way to evaluate the effectiveness of Promise Neighborhoods, a $100 million Obama administration competitive-grant program intended to improve education for students in distressed communities, the Government Accountability Office says.
The GAO report, released Wednesday, explains that the department requires grant winners to collect extensive data on things like individuals they serve, services they provide, and related outcomes, as well as report annually on multiple indicators. However, the department told GAO’s investigators that it needs to conduct a systematic examination of the reliability and validity of the data to determine whether it will be able to use the data for an evaluation.
“While [the department] is collecting a large amount of data from Promise grantees that was intended, in part, to be used to evaluate the program, the [department] offices responsible for program evaluation ... have not yet determined whether or how they will evaluate the program,” the report’s author, Jacqueline Nowicki, wrote.
“While [the department] recognizes the importance of evaluating the Promise program, they lack a plan to do so,” the report continues. “If an evaluation is not conducted, [the department] will have limited information about the Promise program’s success or the viability of the program’s collaborative approach.”
Researchers at the Urban Institute, a non-partisan policy organization that is working with the department to facilitate data collection in the grant program, are considering various evaluation methods, including approaches that estimate a single site’s effect on outcomes and aggregating those outcomes, according to the report.
On the positive side, the GAO report found that the Promise Neighborhood competition spurred grant winners to collaborate with various organizations in order to maximize existing funding and leverage funding from various new places.
Several grant recipients told GAO investigators that the collaboration helped them complete projects during the initial planning phase, which in turn helped build momentum for loftier goals. In addition, the grant winners said that the collaboration yielded important benefits, including deeper relationships with partners and the ability to attract additional funding.
The Education Department launched Promise Neighborhoods in 2010 with $10 million from the federal economic stimulus program. Congress has approved subsequent rounds of funding for the competition that’s been doled out to more than 50 education-focused organizations in more than a dozen states across the country.