Federal Funds 'Well Spent' on Science Effort, Study Finds
The National Science Foundation in the early 1990s began sinking tens of millions of dollars into large-scale, multiyear efforts to improve math and science education in more than half the states.
Now, a team of researchers hired to evaluate that effort, known as the Statewide Systemic Initiatives, has concluded that the money for the most part was well spent.
"For $50 million a year, it was a small investment in federal terms," said Andrew A. Zucker, the researcher who co-directed the project. "For that money, the program garnered a very respectable--really, impressive--amount of program change."
Mr. Zucker, who is also the program manager for mathematics and science education at sri International, a research firm in Menlo Park, Calif., was one of three researchers presenting findings from the evaluation during a meeting here this month of the American Educational Research Association.
The aim of the systemic initiatives was to provide states with a five-year-long infusion of money to make lasting changes in every facet of their math and science education programs. States could decide on their own, for example, whether to concentrate on the state level and write a new framework for teaching those subjects or focus on the classroom and spend more money on training teachers. The main requirement was that the different parts of the system had to work together.
The federal agency began allocating money to states in 1991. Over the course of the program, it supported systemic reform efforts in 25 states and Puerto Rico at a cost of about $266 million through last year.
The effort was not free of controversy, however. The foundation booted Rhode Island from the program in 1994 and withdrew funding for three more states--Florida, North Carolina, and Virginia--two years later. The agency cited problems with the management and design of those states' reform efforts. But officials in some of the states whose funding was pulled complained that the NSF, which had put a succession of managers in charge of the national initiative since its inception, had given them contradictory feedback. ("NSF Cuts Off Funds to D.C., Three States," Oct. 2, 1996.)
In such states as California and Virginia, the curricular changes that reformers were pushing in math ran into political opposition.
Even so, the team of researchers that conducted the $4.6 million, NSF-financed evaluation said the systemic-initiative program left behind some lasting improvements.
Each participating state and territory, for example, created frameworks for teaching math and science before its money ran out. And, across the states, those curricular visions were similar and rooted in the math and science standards that were just emerging at the national level at the time.
A few states also created permanent institutions for math and science reform, such as regional professional-development centers.
One-third of the dollars the states spent went to improving teacher training, often through two- to six-week science and math academies held in the summer. And the evaluators judged those efforts to be of "high quality."
But at the classroom level, the researchers said, states' efforts often fell short. The professional-development programs, for instance, reached no more than 10 percent of any state's math and science teachers in any one year.
And even among that group of enthusiastic volunteers, many experimented only sporadically with the new approaches they had learned, according to the evaluators' report.
State reformers also made few inroads into colleges and universities, where the next generation of math and science teachers was being prepared. Improving teachers' preservice education was a primary strategy in only two states and was used as a secondary strategy in 12 others, said Margaret E. Goertz, a co-director of the Consortium for Policy Research in Education, a Philadelphia-based federal research center that also took part in the evaluation.
And few states documented improvements in student achievement that might have resulted from the program. But six of the grant recipients that were able to point to improving test scores have been given another five-year round of funding by the NSF. They are: Connecticut, Louisiana, Massachusetts, Texas, Vermont, and Puerto Rico. The list may yet grow, agency officials said, because five other states are in their last year of program funding and could reapply for grants.
Luther S. Williams, the NSF's assistant director for education and human resources, characterized the evaluators' assessment of the program as "reasonable."
"The whole effort was viewed on our part as an experiment," he said. "But we are very encouraged by the states that have done an exemplary job."
But other researchers at the meeting said the findings also raise a crucial question about evaluating any systemic reform: how to tell which improvements are due to which change. California, for example, began drafting a new generation of curriculum frameworks long before most other states did--and before it joined the systemic-initiative program.
In their report, however, the researchers tried to take into account reform efforts already under way and measure the added boost that came from the systemic-initiative program, said Ms. Goertz. Moreover, she added, "in 1991, very few states had frameworks in place."
Individual case studies of states, created as part of the five-year evaluation of the initiatives program, are available on the World Wide Web at www.sri.com/policy/cehs/edpolicy.html.