Corrected: An earlier version of this story incorrectly identified Marguerite Roza’s affiliation. She is a research associate professor at the University of Washington.
Details are starting to emerge about a study planned by the U.S. Department of Education to determine what states are doing with their education money under the economic-stimulus law and whether the efforts being funded improve schooling.
“Most of the money is really for stabilization, but $10 billion is going to be for competitive grants,” John Q. Easton, the director of the department’s Institute of Education Sciences, told his advisory board at its Nov. 9 meeting. “I certainly don’t want to be here in three years and have somebody say, ‘What did we get for that $10 billion?’ ”
“We’ve got to be learning from this,” he added of the American Recovery and Reinvestment Act funding. “It’s too huge an opportunity to lose.”
The $787 billion package includes some $100 billion to help bolster education. The first reports show that the biggest portion of such aid spent so far has gone to save or create jobs. (“Stimulus Data Trove Examined,” Nov. 11, 2009.)
The institute is the lead office on the research project, which will also involve the department’s office of policy, planning, and evaluation.
Sue Betka, the IES’ deputy director for administration and policy planning, said the institute plans to spend $35 million over the next two years to get the evaluation up and running. The study is expected to last at least five years, as researchers sort out whether the strategies states put in place using stimulus money are making a difference in the quality of education.
As it stands now, the research design calls for “a three-layer” approach, according to Mr. Easton. The first layer will look at how states are using the money; the second, at whether common strategies are emerging; and the third calls for embedding impact evaluations within those cross-cutting strategies to determine if they’re producing results.
To get the answers, the Education Department plans to conduct nationally representative surveys, as well as smaller, more frequent, and more focused polls, to help it “figure out midcourse corrections and give feedback to states on what’s happening in their states,” said Marsha Silverberg, the acting commissioner for knowledge utilization at the IES.
For some programs, such as the $4 billion state competition under the Race to the Top Fund, the department will survey every grant recipient. “We’re also trying to reduce the burden on states by trying to gather as much information as possible from the required reporting that states will have to do,” Ms. Silverberg said.
Randomized experiments and correlational studies are among the other kinds of research methods the department hopes to use in its evaluation.
Final details are not expected until early next year; data collection will begin by fall 2010. The department’s policy, planning, and evaluation office, meanwhile, has contracted with the Washington-based American Institutes for Research to conduct in-depth studies of the stimulus-fueled improvement efforts unfolding in a small set of low-performing schools.
Good Timing
The pace and the design of the work are winning high marks from some experts.
“If you wait until programs have already started, it’s virtually impossible to get reliable information on what their effect was,” said Eric A. Hanushek, a researcher at the Hoover Institution at Stanford University and the president of the National Board for Education Sciences, which advises the IES.
The institute has recruited a group of scholars for advice on an evaluation strategy. They include: Dale Ballou, an associate professor of public policy and education at Vanderbilt University; Kevin Carey, the policy director of Education Sector, a Washington think tank; Tom Cook, a professor of sociology, psychology, education, and social policy at Northwestern University; and Ronald F. Ferguson, a senior lecturer in education and public policy and senior research associate at the Malcolm Wiener Center for Social Policy at Harvard University.
The others are: Adam Gamoran, a professor of sociology and educational policy studies at the University of Wisconsin-Madison; Margaret E. Goertz, an education professor at the University of Pennsylvania; David Heistad, the executive director of research, evaluation, and assessment for the Minneapolis public schools; James J. Kemple, the executive director of the Research Alliance for New York City Schools; Susanna Loeb and Sean F. Reardon, both associate professors of education at Stanford University; and Marguerite Roza, a research associate professor at the University of Washington.
