Facts Over Fads
Testing Out 'Innovations' Might Save Money, Time, and Good Will
Educators at all levels, from classroom teachers to national policymakers, routinely use and approve materials and techniques without testing or evaluating them. This costs schools millions of dollars and does not yield the results educators and the public are clamoring for. Consider the following occurrences, all too common in American education:
- The federal government decides to terminate the two federal programs in education that require research-based practices (this almost happened this fall with the National Diffusion Network and Follow Through).
- A state government imposes new requirements without studying whether the requirements will result in any improvement in learning.
- A textbook company spends half a million dollars in the name of research--actually market research--yet manages not to study whether the book in question is effective in helping students learn.
- A school district announces a new curriculum for September that has not even been written, let alone tested, by August.
- A district, working with a college of education, creates an elaborate teacher training program to teach methods that have never been proven to work.
If the world of science operated this way, scientists would give Newton's experiments in alchemy the same weight as his law of gravity. One difference between science and education is that in science, new knowledge is acquired through testing, and then published and verified by the research and replication of results by other scientists. It is only after a careful process of research, experimentation, and verification that a new approach gains the status of an accepted part of scientific practice; those that fail are discarded. But in education, untested fads sweep through the profession, gathering authority by the number of schools using them, not by proven gains in learning. The field does not distinguish between innovations which merely create change, and reforms which are changes that yield improvements in student achievement.
This unscientific approach to education squanders the time, money, and effort of those trying to improve learning outcomes for students. The enthusiasm and energy of new teachers quickly become transformed into dissatisfaction and cynicism when they are caught up in a cycle of requirements to use untested innovations that, ironically, replace older failed changes. Veteran teachers find themselves labeled "afraid of change'' and "burned out'' when they recognize some of today's reforms to be re-labeled experiments that have already failed in the past. At the same time, the public is growing tired of "throwing money'' at the schools to pay for new reforms when most do not improve student knowledge or test scores. As schools lurch from one untested instructional method or tool to another, we ultimately undermine the education system and academic performance.
It is no wonder teachers do not think the reforms of the last decade have been effective in improving the performance of all students. A recent survey conducted by the federally funded National Center to Improve the Tools of Educators, or NCITE, found that three-quarters of the teachers surveyed (76 percent) believed school reforms had not benefited the students who need help the most. The survey results indicate that teachers want facts, not fads, to govern the selection of new reforms and materials. Respondents said they want teachers, who see the results of methods and materials with students, to decide what purchases to make. And they want more training to enable them to make the best use of new techniques.
The teachers' solution is to use everyday common sense; they said that schools should scientifically test reform innovations and only use the ones that can produce the results that are claimed for them. While over half (59 percent) of the teachers said schools should use extensive classroom research to decide the effectiveness of an approach, fewer than one in 10 (9 percent) said they believe schools actually do this.
Particularly important, the teachers said, is testing educational materials before using them in the classroom, since these have a major impact on student learning. Virtually all (90 percent) of the 309 billion hours Americans spend in schools involves the use of textbooks, activity guides, films, or other materials, according to an Education Product Information Exchange study. These tools and practices influence between 75 percent and 90 percent of what goes on in classrooms. If these materials are not of high quality, students will not learn as quickly or as well.
Teachers, schools, states, and the federal government can use a process for appraising educational innovations and judging their merits. Examining these tools and materials before they are used in schools would insure that mediocre materials and unworkable reforms do not enter the classroom, saving teachers, students, and administrators from failure and wasteful expense. Innovations should be selected for wide-scale adoption only when they have been proven effective. If that is not possible, educators should arrange for a small-scale adoption to fieldtest the new approach to see if it has had a positive effect on student achievement. When selecting an approach, educators should determine whether the approach and its outcomes are clearly defined. They should investigate instructional research on the effectiveness of the change. An accountability system must be developed to judge the results of the innovation. In addition, the approach must be manageable, equitable, and reasonably priced.
Once an approach is selected, the school system must carefully plan its implementation. This planning should involve a wide range of stakeholders, including teachers, parents, administrators, community members, and the school board. If this is a valid reform, the planning of its implementation will persuade other people to support the reform. For instance, one principal convinced her staff to adopt a new curriculum by arranging for staff visits to a successful school that was already using the curriculum. Students should be tested before the reform is implemented in order to establish a basis for measuring the reform's effects. The school must provide staff development for the teachers and administrators who will use the new method. In addition, all the different sources of authority within a school and district must cooperate on the goal.
Educators should not just assume the new approach will work. It should be monitored by a series of ongoing assessments to insure that children are learning and that the approach is being implemented. There should be an alternative plan for adjusting instruction based on student performance. For example, one district in Michigan has an instructional facilitator in each school who tests students and works with teachers to adjust instruction. These tests should be tailored to measure what is being taught; standardized tests frequently fail to measure the new method or material adequately. Program criteria should be set in advance and the final result compared to the expected one. Students' performance should be compared with their level before the approach was implemented and with that of similar children in schools without the new approach.
It costs no more to use a tested reform than an untested innovation. But unproductive innovations not only waste money but also the effort of school personnel and the hard work of children trying to excel. While in the short term they may distract critics who demand that school officials produce an immediate solution to the problems of the schools, in the long term these empty promises do nothing but discredit education and its practitioners.
Vol. 13, Issue 14, Page 40Published in Print: December 8, 1993, as Facts Over Fads