‘Toolbox’ Expected to Help Assess After-School Programs in California

By Nora Fleming — March 11, 2011 3 min read

Holding after-school programs accountable can be a challenge, according to some after-school leaders. Difficulties retaining staff and a lack of concrete assessment objectives have left many programs without the tools to evaluate themselves.

Programs receiving grants from California’s After School and Education Safety Program (ASES) are held to some accountability measures that can factor into whether their grants are renewed. The ASES program, which spends $550 million annually supporting after school at 4,000 schools in the state, requires sites to submit the attendance records of participants and some method to evaluate student outcomes, either standardized-test scores or a measure of student-behavior change, homework completion, or skill development.

Because standardized scores are more readily accessible, in the past, most sites have submitted test scores as a measure of student outcomes, said Deborah Lowe Vandell, chair of the education department at University of California, Irvine. But the state department of education is currently supporting efforts to provide sites with other measures to assess the impact ASES programs have had on youths.

Lowe Vandell is now working on a project funded by the state department of education and the David & Lucille Packard Foundation called the California Afterschool Outcome Measures Project (CAOMP). It will assess, through an online data system called the “online toolbox,” whether after-school programs have affected student performance and behavior.

Sites can opt to submit student self-surveys or staff or classroom-teacher reports on student performance (or all three) for assessment. The survey questions, which ask, for example, about student persistence, independence, and socialization, were created after reviewing literature on what student behaviors were indicators of improvement that could be influenced by a quality after-school program.

“There are other ways to look at student outcomes other than standardized-test scores,” Lowe Vandell said. “As we start to measure additional behaviors consistently and reliably and they are validated against other outcomes we care about, they can help develop a broader array of measurement tools that drive [after-school] programming.”

Last fall, the outcome-measures project collected data through the online toolbox from 157 sites, totaling 6,000 students, 4,300 staff members, and 670 teachers. Spring data will be pulled next month, and by summer, programs will be provided overall report assessments on positive/negative changes in their student outcomes over the school year. Sites can see how their assessments stack up against their site peers throughout the state, and use the data to fulfill the accountability requirements requested by the state department. Project leaders hope to open the toolbox to after-school programs nationwide by the fall, though will eventually charge a modest fee for usage.

The California AfterSchool Network, an organization that works to support state after-school programs and favorable out-of-school-time policies, sees a link between student outcomes and the general quality of after-school programs.

Since the rapid expansion of after-school programs in California as a result of the grant program, many new sites have struggled with having the know-how to assess the quality of their programs and improve them, said Jeff Davis, a program coordinator with the California AfterSchool Network. The data currently required for the state department of education examines more general student-performance markers and less program quality components that can lead to improved student outcomes, he added.

With collaboration among state after-school leaders, the California AfterSchool Network has developed an online “Program Quality Self-Assessment Tool” that will help with a staff-directed evaluation process. The tool helps programs assess themselves on 11 criteria they deemed important to deliver a well-rounded program by measuring how well programs are meeting those objectives in tiered levels. Categories include community partnerships, school alignment, staff professional development, diversity and inclusion, and English-learner support.

Most of the criteria evaluators are applicable to after-school programs throughout the country, Davis said, other than the English-learners objective, which is more geographically specific.

Read more about after-school programs in California in my article this week on LA’s BEST, a Los Angeles-based after-school organization that receives significant support from the ASES program.

A version of this news article first appeared in the Beyond School blog.


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Future of Work Webinar
Digital Literacy Strategies to Promote Equity
Our new world has only increased our students’ dependence on technology. This makes digital literacy no longer a “nice to have” but a “need to have.” How do we ensure that every student can navigate
Content provided by
Mathematics Online Summit Teaching Math in a Pandemic
Attend this online summit to ask questions about how COVID-19 has affected achievement, instruction, assessment, and engagement in math.
School & District Management Webinar Examining the Evidence: Catching Kids Up at a Distance
As districts, schools, and families navigate a new normal following the abrupt end of in-person schooling this spring, students’ learning opportunities vary enormously across the nation. Access to devices and broadband internet and a secure

EdWeek Top School Jobs

Speech Therapists
Lancaster, PA, US
Lancaster Lebanon IU 13
Elementary Teacher
Madison, Wisconsin
One City Schools

Read Next

Education Briefly Stated Briefly Stated: January 13, 2021
Here's a look at some recent Education Week articles you may have missed.
8 min read
Education Obituary In Memory of Michele Molnar, EdWeek Market Brief Writer and Editor
EdWeek Market Brief Associate Editor Michele Molnar, who was instrumental in launching the publication, succumbed to cancer.
5 min read
Education Briefly Stated Briefly Stated: December 9, 2020
Here's a look at some recent Education Week articles you may have missed.
8 min read
Education Briefly Stated Briefly Stated: Stories You May Have Missed
A collection of articles from the previous week that you may have missed.
8 min read