Despite the hundreds of times scientifically based research (and its variants scientifically based reading research and research based) appears in No Child Left Behind I, it has not been part of the general debate around NCLB II. It has yet to be mentioned by either Presidential contender as they explain their views on the federal role in public education. In my view the concept of program evaluation represented by these terms is the most important issue in federal education policy the candidates don’t discuss.
Why is it important? Why isn’t it part of the debate? And is it inevitable? If a candidate decides to take a position, what are the options and are some more favorable to one candidate or another?
Today, I’ll take a stab at the first three questions.
Why is SBR important? Edbizbuzz readers know my general argument. Despite the prominence of scientifically based research and its spinoffs in the law (which for simplicity here I will label SBR), Department of Education practice has given demonstrations of program efficacy no meaningful role in the implementation of No Child Left Behind. To get an idea of 1) the Department’s oversight of the market in instructional products and services aimed at the classroom, and 2) the implications of this failure, imagine the sale of pharmaceuticals and health aids in the absence of the Food and Drug Administration,
Circa 1900 the national press publicized a series of deaths involving the addition of unsafe compounds into consumer health products. Progressive Teddy Roosevelt made drug safety part of his 1904 campaign’s Square Deal and signed the Pure Food and Drug Act in 1906. In 1938, again following the publicity surrounding a series of deaths associated with sale of patent medicine for inappropriate purposes in an inappropriate form, his cousin Franklin signed the Food, Drug and Cosmetics Act. That law requires that sellers’ claims have some basis in fact. This and subsequent federal legislation offer the public a reasonably high level of assurance that the health products they purchase will at least do no harm and probably ameliorate the health issue they are purported to address. The 1906 Act took the cocaine out of Coca Cola. With the 1938 law, the product began its transition from the medicine cabinet to the kitchen refrigerator.
Demonstrating that a product will first, do no harm and, second, meet advertised claims, involves policy judgments based on some degree of scientific knowledge. Decisions about the process to assure product safety and the level of proof required were much less sophisticated at the turn of the last century than they are today. Even now there is a debate over the standards for permitting pharmaceuticals to enter the market. Fast Track drug approval helps get important advances to more patients, faster. The problems with VIOXX demonstrated how drug trials can miss important negative effects on particular patient subgroups. Getting the balance right is a perpetual challenge. Nevertheless, the state of the evaluation art would not have advanced very far if the federal government had not got started in 1906.
At the turn of this century, the market in products and services to improve teaching and learning in the classroom is not all that different from the market in patent medicine of 100 years ago. The evaluation of programs designed to improve student learning as measured by test scores was virtually unheard of before the 1990’s and remains something like the opposite of common practice. Very few programs can demonstrate educationally significant improvements in student performance on state tests. Some programs remain on the market despite the fact that third party evaluations suggest they actually retard student learning. Despite the fact that the tools for assessing the efficacy of educational programs now are at least as sophisticated as those applied to drugs in 1906 and No Child Left Behind at least permits (and I and other would say requires) program evaluation, the U.S. Department of Education does not regulate the safety or efficacy of the products and services purchased to give our children the skills and expertise necessary to become informed citizens, productive workers, and fulfilled individuals. Even more scandalous is the fact that through the Department, federal taxpayers provide billions of dollars to subsidize the purchase of these programs by state and local education agencies.
And while the problem of evaluating educational programs is arguably more complicated than drugs, the state of the art has advanced considerably since 1906. Certainly we have come to a point where it is possible to identify harmful programs, and to distinguish education from edutainment. And one sure way to advance the state of our evaluation art in educational programs is to start making it important.
Why isn’t SBR part of the debate? The argument against making “safe and effective educational programs” an issue in the Presidential campaign may find its source in a fear of taking on the multinational publishers who still benefit from a market unfettered by evaluation, a lack of interest or insight on the part of candidate’s education policy advisors, a sense that the theme would be met by deaf ears up their chain of command, or something else, but the reasons I’ve seen and heard boil down to 1) a lack of public interest and 2) complexity.
I reject the complexity outright. Once drug safety became a public issue in the early 1900s, complex decisions about evaluation processes and criteria were secondary to the debate. The political question was whether we needed regulation and, later, whether it was actually protecting the public. The technical details did not get in the way of making drug safety important – public awareness of the consequences of the status quo was the key.
Up to a point, I buy arguments centered on the lack of public interest. Popular pressure for safe drugs was aroused when muckraking reporters publicized the worst examples of drug makers’ disregard for public health, and progressive politicians joined the crusade. Whether caused by harmful substances or empty promises, death has a way of focusing attention on a problem. Unfortunately, the contribution of faulty educational programs to wasted lives - for example, that portion of the 50% of African-American males who fail to graduate high school and wind up on the wrong end of a gun, let alone the general economic and personal consequences of mediocre schooling - is not so obvious or direct.
Still, there is the progressive/muckraker factor. I don’t think public interest arises solely from the ground up; others light the fire, fan the flames, and channel the conflagration. My sense is that historians agree on the importance of people like Upton Sinclair, Samuel Hopkins Adams, Florence Kelley and Harvey W. Wiley to passage of food and drug legislation. Theodore Roosevelt’s role suggests the place for political leadership today.
However, judging from the reactions of Congress to the Reading First fiasco and press reports on studies of the programs offered by Supplementary Educational Service, I will concede: First, that policy wonks, educators and reporters are so poorly educated in evaluation that they do not understand the difference between statistical and educational significance. Second, that providers, their trade groups and their policy advocates would prefer that the difference remain obscure.
Is it inevitable that SBR remain unadressed? Returning to the theme of leadership, one thing I find instructive from the passage of No Child Left Behind I was the ability of candidate and President Bush to make the unsatisfactory performance of African-American and minority students a national disgrace rather than their disgrace, and to shift a fair share of the blame from those groups to our system of public education. The combination of “leave no child behind” and “soft bigotry of low expectations” was a brilliant merger of policy and politics. Its basic truth resonated with policymakers and the general public. It connected a conservative President and American business directly to the nation’s least advantaged groups. It split the Democratic Party’s traditional labor-minorities alliance, and forced Democrats in Congress to choose African American’s and Latinos over teachers unions, at least temporarily isolating the AFT and NEA. By distilling the problem into two simple phrases, Bush made complex changes in practice politically relevant, in the same way that needless deaths legitimized the regulation of pharmaceuticals. Absent those simple phrases, I very much doubt that the arcane provisions moving accountability from school-wide averages to individual student performance within subgroups would have become federal law.
If I were a gambling man, I would not bet in favor of the candidates tackling SBR, but this outcome is not inevitable. It would take a degree of interest and expertise in the market for school improvement programs that I do not see now in either campaign. To the extent that such interest and expertise can be found in campaign staffs, it is quite unlikely that campaign directors will be all that interested - unless public education policy were to become far more important to winning the general election. In my estimation that would only come about if African-American and Latino votes prove to be crucial when the Presidential election is viewed from the perspective of the Electoral College. More on this later.
Next: If a candidate decides to take a position, what are the options and are some more favorable to one candidate or another?
Marc Dean Millot is the editor of School Improvement Industry Week and K-12 Leads and Youth Service Markets Report. His firm provides independent information and advisory services to business, government and research organizations in public education.
