Education Chat

Chat Transcript: From Research to Practice

This Live Chat discusses "scientifically-based" education research and how it can be implemented with qualitative data to provide meaningful insights as to what is happening in the classroom.

Lisa Towne:
Certainly there is a tension between what we know and what resources are available and the requirements of No Child Left Behind with respect to the scientifically based research provisions. But evidence-based eduacation is not just about complying with NCLB; it’s about creating systems and instructional environments that seek out, read, and reflect on research on a regular basis as it pertains to policy and practice. I’m not minimizing what states, districts, and schools are experiencing right now given the time crunches and funding implications associated with NCLB, but I would argue that in the long term we can get these systems set up such that compliance with respect to SBR is non-sensical; it’s just a way of doing business.


Question from Patricia Pulver, professor of education, Keuka College, NY:
When teachers do research in their classrooms (action research) it is by its very nature contextual and not generalizable. How will this type of research be added to the body of knowledge? I would hate to see a system developed that does not respect the experiences and insights of classroom teachers.

Lisa Towne:
I would agree that such research ought to be a part of an evidence-based education system. The real trick is integrating what is known from more systematic research with the localized insights generated from action research as well as any number of other sources that help contextualize issues for local circumstances. Both types of research have strengths and weaknesses; the key is recognzing the role of each and figuring out ways to link them in a coherent way that can powerfully inform improvements in educational practice.


Question from Deanna Woods, retired teacher, NWREL:
What kinds of supports does IES plan to provide to ensure that teachers can master the instructional practices most effective for a diverse student body?

Grover “Russ” Whitehurst:
With the exception of the Regional Educational Laboratories, which are administered within IES, IES has no direct responsibility for providing professional development and technical assistance to teachers. Many such activities are supported by federal funds that flow through other offices of the Department.


Question from Thomas Crochunis, Research and Development, Brown University:
Given that there are many issues unresolved about how best to transact (transfer, apply, negotiate) new evidence-grounded knowledge between research and practice, are there plans for funding research into this process at the Department of Education?

Grover “Russ” Whitehurst:
We are very interested in funding research on knowledge utilization. The Interagency Education Research Initiative, a joint project of IES, NSF, and NICHD, has funded many projects that address issues of scale-up of research to practice.


Question from K. Dawn Marotta, Parent, Massachusetts:
As a parent of a child with a reading disability, I feel my head swimming as I read each new question and answer. What does scientific research mean for parents and their children? How do parents now if the program their school is using is effective, researched, and appropriate? It seems that many parents are getting the run around on what is truly researched and what is not.

Grover “Russ” Whitehurst:
I hope that the products of the What Works Clearinghouse will be useful to parents.


Comment from Russell Mays, Associate Professor (Ed leadership) University of North Florida:
YES!! I am completely in favor of “creating systems and instructional environments that seek out, read, and reflect on research on a regular basis as it pertains to policy and practice.” Getting that to the real world is the challenge. We have too many people fearing for their jobs and / or reputations and at present are giving too little attention to the real task of teaching and learning. Thank you for the insightful information this afternoon.


Question from Richard Layman, Superintendent, King & Queen County Public Schools, Virginia:
Few educational programs have been tested empirically for implementation in rural areas, especially in small, high poverty school districts located in rural areas. How does the USDE plan to assist school districts in finding programs that work in rural schools and their communities?

Grover “Russ” Whitehurst:
We have recently announced our plans to fund a research and development center on rural education. The National Center for Education Statistics, within NCES, provides a website of resources in rural education.


Question from Pier Junor, Assistant Professor, Georgia State University:
Before entering schools or districts to conduct research, isn’t there a policy where the researcher has to go through a rigorous review at their institution before conducting the actual research? If that is done, then how accountable is the researcher to the proposed study? Who monitors that process?

Lisa Towne:
Yes, the process is typically monitored by institutional review boards which are normally housed in universities. IRBs are concerned with ensuring that the rights of the research participants are upheld and that research plans include appropriate ethical safeguards.

Our guests include:

Grover J. (Russ) Whitehurst, director, Institute of Education Sciences. The Institute, part of the U.S. Deparment of Education, conducts, supports, and disseminates research on education practices that improve academic achievement; and

Lisa Towne, senior program officer, National Research Council, National Academy of Sciences.


Debbie Viadero, Education Week (Moderator):
Welcome to Education Week on the Web’s Talk Back Live. Today, we’re discussing “scientifically-based” education research with Grover “Russ” Whitehurst, the director of the U.S. Education Department’s Institute of Education Sciences, and Lisa Towne, the senior program officer overseeing a series of National Research Council committees that have been examining scientific inquiry in education. I’m Debra Viadero, an associate editor here at Education Week, and I’ll be your moderator.


Question from Debbie Viadero, Education Week:
Let’s start out with a question for each of you. Mr. Whitehurst, what does the education department mean by “scientifically-based” research?

Grover “Russ” Whitehurst:
The Institute of Education Sciences supports research that uses conceptual models, research designs, data, statistical analyses, and logical inferences that are appropriate to the questions addressed and that support the conclusions drawn. Particular methods and approaches may be appropriate, i.e., “scientific,” for some questions and conclusions, but inappropriate or unscientific for other questions and conclusiosn.


Question from Debbie Viadero, Education Week:
And Ms. Towne, how does the federal government’s definition differ from what the National Research Council is calling “scientific inquiry” in education?

Lisa Towne:
The federal government has a couple of definitions of “scientifically based research,” most notably in the No Child Left Behind Act and the Education Sciences Reform Act. These two defintions are not the same, and they differ still when compared to the scientific principles that are articulated in the NRC book “Scientific Research in Education” (see www.nap.edu).

These differences can be attributed at least in part to the different purposes for which each definition was devised. Although Dr. Whitehurst will elaborate on NCLB and/or ESRA, I’ll describe briefly the purposes of these definitions and those that appear in the NRC report to illustrate this point. The NCLB definition is meant to give states and districts implementing that law a way to judge the (existing) scientific evidence with respect to the effectiveness of various educational interventions that would be supported with federal funds. The ESRA definition, in contast, is meant as a kind of definition for scientific quality with respect to the kind of (future) research that the Institute of Education Sciences funds in its research competitions. Thus, I would argue that one reason why the ESRA definition is broader than the NCLB definition is because the purpose of the ESRA provision is broader than the NCLB provision.

In the NRC book, the purpose was to provide a broad articulation of the nature of scientific inquiry into questions of teaching, learning, and schooling. The National Educational Research Policy and Priorities Board (the policy board of the former OERI) asked the NRC to do this because defintions of the sort that appear in NCLB and ESRA were popping up in statutory language as early as 1995, and they wanted a group of scholars from a range of fields to articulate the nature of science and how it applies to the systematic study of educational problems. Thus, it was not tied to the specifics of federal programs, compliance issues, or research policy, and therefore is broader still than either the NCLB or ESRA definitions.

One of the specific ways in which the NRC principles differs from both the NCLB and ESRA definitions is with respect to randomized field trials--the NRC book, consistent with the purpose I’ve described, places these methods in the context of how a wide range of research methods can successfully be employed to address particular research questions and the role of methods in scientific inquiry more generally. In fact, one of the most important conclusions in the NRC book is that the choice of method must be driven by the nature of the question posed.


Question from Diane DeMott Painter, Ph.D. Technology Resource Teacher, Fairfax County Public Schools, VA:
How do you view practitioner research (Teacher Research/Action Research) in terms of providing rich, authentic insights into what is happening in classrooms?

Lisa Towne:
The central concept of “evidence-based education” (and many other evidence-based applications in other fields) is that findings from rigorous research would be integrated with the professional expertise and judgment of educators and administrators to run schools and deliver instruction effectively.

This notion means that, for example, teachers would be supported to be intelligent consumers of peer-reviewed research, to reflect on how these findings relate to current practice and professional judgment, to participate in and contribute to research in their schools. So I think that engaging in “action research” to get a real feel for the power of research and to deepen insights into phenomena as they play out in localized contexts, when done well, can make a substantial contribution to a truly evidence-driven educational system.


Question from Jan Greenberg, Early Childhood Training Coordinator, Reading Is Fundamental:
Please address the role and strengths of well-done qualitative research, how to evaluate qualitative research studies, and how both quantitative (including scientifically-based) and qualitative research findings can be used together to inform instructional practice.

Lisa Towne:
The NRC report articulated principles of scientific research that apply to both quantitative and qualitative research. The committee felt strongly that the dichotomy between the two was overemphasized and counterproductive to understanding high quality scientific research. So both need to have clearly specified, researchable questions; link data and theoretical concepts; match methods to questions; engage in a logical chain of reasoning that identified counterhypotheses; and the work is open for profesional scrutiny and constructive critique.

It also showed lots of examples of how quantitative and qualitative research fit together in a line of inquiry to strengthen inferences. Generally, when a finding can hold when subject to multiple kinds of data and methods, it is stronger than if it is subject to one alone.


Question from Kenneth Goodman, Retired reading researcher, University of Arizona:
In view of the composition of the National Reading Panel and interpretation of its findings built into the enforcement of NCLB isn’t this more proof that concerned scientists are right in saying that this administration uses its political agenda to define what is rigorous research?

Grover “Russ” Whitehurst:
I was not involved in the National Reading Panel, but since it was formed in 1997 and issued its report in 2000, there would seem to be a temporal problem in linking its composition to the current administration.


Question from Christopher Brown, Pearson Education:
Does IES support the NRC’s Strategic Education Research Partnership and what is its status?

Lisa Towne:
The “SERP” initiative has been under development for several years and has included a range of federal and private/non-profit funding sponsors, including OERI/IES. Right now, a small steering group is working to actually implement the major recommendations for developing an infrastructure that systematically links research and practice. The plan is for it to consist largely of a compact of states and be overseen by an independent 501c3 organization.


Question from Jose Vazquez, New York University:
For Lisa Towne, The suggestions for scientific research in your book are not very clear, i.e. examples lack some important data. How is scientific research supposed to improve the quality of educational research? Can you be more specific?

Lisa Towne:
First, a clarification. The committee did not say and does not believe that scientific research is meant to improve the quality of education research. As described more fully in the NRC book, scientific research is one kind--a very important kind, but not the only--of education research. Historians and philosphers of education, for example, make substantial contributions to our understanding of education, but for the most part their scholarly work would not be considered scientific.

So in general there are two distinct adjectives that can be applied here to education research: quality and scientific. Some education research is high quality, and some is not. Some education research is scientific, and some is not. Further, scientific research can be high quality or low quality, just like historical research (or other research that is not done in the scientific tradition) can be high quality or low quality.

All that said, the NRC book focused exclusively on defining scientific inquiry in education. As for the examples that appear in the book, the committee chose several scientific studies as well as lines of inquiry in education and other fields to illustrate several key ideas. Taken together, they show how the principles hold across various fields, they show how the quest for scientific understanding in any domain is a long-term, ongoing process in which what we think we know at one point in time can be turned on its head a short time later, and they show that multiple methods are needed to fully inform complex questions.

Finally, another issue that I think is embedded in your question is that of the relationship between quality research and its utility in improving education policy and practice. This is a pretty complex matter, but very succinctly, ensuring high quality (scientific) research in education is a necessary but not sufficient condition for improving policy and practice. Attention to how information is shared, used, understood, and integrated into educational decision-making and day-to-day practice is in many ways a tougher nut to crack than defining quality, and the NRC report we are talking about very deliberately does not attempt to tackle such issues.

On that note, however, there is another NRC report called the Strategic Education Research Partnership that conceptualizes very broadly a system of bringing high quality research to bear more systematically on educational practice. You can find that on www.nap.edu.


Question from Thomas Crochunis, Brown University:
What existing theoretical concepts and empirical research from fields other than education (such as organization science, health science, etc.) seems promising, or at least worth considering, as we go about seeking to improve knowledge utilzation, application, and refinement processes? Do any particular studies or names come to mind?

Lisa Towne:
There are certainly lots of fields and individuals that could inform a fully functioning evidence-based education system. Certainly evidence-based medicine is one. This has a fairly large literature, but the seminal article is widely viewed to be Sackett et al, 1997. There are important lessons to be drawn from evidence-based medicine (itself not as mature a trend as one would hope or expect and struggling with many of the same issues we deal with in education) but understanding the nature of education research specifically will also be important in figuring ways to use it effectively in educational settings--here i would point to philosophers of education (my favoriates are Denis Phillips and Bob Floden) and some economists like Dick Murnane and Dick Nelson. Finally I would point to Carol Weiss’s work at Harvard on the ways in which various forms of information, including research, are used in social policymaking as extremely illuminating.


Comment from Bill Harshbarger, Teacher, Mattoon High School:
Research must be true. Teachers aren’t interested in contradictory conclusions. Research must be practical. Not only must a new discovery work, but it must work with my kids in my school. The great break through in school improvement won’t come from research, it comes from teachers-- those who have time to experiment with new ideas and are able to incorporate them into the high-pressure, daily routines of preparing lessons and teaching.


Question from Jack Fretwell, Owner, Starboard Training Systems:
How can effective solutions escape the stigma of “agenda-driven” once they’ve become productized? New ideas from businesses face a formidable wall of resistance from the academic community. Small businesses with meager funding have little hope of success. Yet this a source of excellent ideas. Will Lisa Towne’s proposed agency act as a clearinghouse for cooperation between business and academia supporting both research and commercial application?

Lisa Towne:
If you are referring to the Strategic Education Research Partnership, I can say that yes, the insights from and cooperation with businesses has been part of the plan all along. There are indeed important connections to be made here.


Question from Michael Paul Goldenberg, Director, RationalMath, LLC:
There appears to be evidence that there are major biases at the Department of Education as to what comprises best practices in literarcy instruction, namely “phonics-only” programs. Given the recent criticism from a dozen Nobel laureates of the Bush administration’s proclivity for distorting and censoring scientific input for policymaking in ways that further its own political agenda, why should parents, teachers, administrators, and other educational stake holders trust that the definitions of “scientific research” and “rigorous evidence” used to evaluate theory and practice will be any less politicized when it comes to education?

Grover “Russ” Whitehurst:
The relationship between research and practice or research and public policy is frequently uneasy and sometimes discordant. This is nothing new. The mission of the Institute of Education Sciences is to support rigorous research that is relevant to education policy and practice, and to encourage its utilization. Our activities are required by statute to be objective and free of partisan political influence. I and the committed staff of career civil servants and excepted service employees at IES work very hard to carry out that obligation.


Question from Rebecca Reevs, Education Consultant, Atlanta, GA:
Under what conditions can a school use a new practice idea that has not been subjected to rigorous testing yet? Does evidence based education preclude the use of untested ideas altogether? Is trial or exploration of new practices unaccetable?

Grover “Russ” Whitehurst:
Evidence-based practice requires development of new programs and practices. We expect and encourage a cycle of development, evaluation, further development, and evaluation.


Question from Dr. Matthew Delaney, NBCT, Curriculum Coordinator/Electives, Whitman-Hanson Regional High School:
What kinds of professional development challenges does the scientifically based reseach mandate hold for current teachers and administrators, who had little experience in creating, selecting, and applying scientifically based research? Furthermore, how can the institutions of higher education (IHEs) charged with the education of the next generation of teachers be encouraged to better prepare their students to be effective designers and critical consumers of scientifically based research that positively impacts education in the classroom?

Lisa Towne:
It is certainly true that for evidence-based education to truly take hold, the nature and findings of research will need to be better integrated into professional development programs, including those in IHEs. Part of this will need to come from better connections between what is being taught in prek-12 and IHEs (the overall lack of such connections being a barrier to a whole host of things, not just the research-to-practice issue). Part of it has to do with upgrading what we expect of teachers and what we expect and trust they will do as professionals. We need to build research into trainng (and ongoing professional development throughout their careers) in a way that empowers them to use research to better their practice but doesn’t strip them of professional judgment.

A final point--as a long term matter, I don’t think we should be asking teachers and other educators to do the lion’s share of the work associated with identifying high quality research and what the findings suggest. For that, we need high quality research reviews (like those that will eventually appear in the What Works Clearinghouse, for example), which take considerable skill and are really the responsiblity of the research community. In the shorter term we need to get educators up to speed on these issues quickly because such resources are lacking.


Question from Sergio Rojas, Educational Consultant, Independent:
There seems to be a study for every thing that is being marketed to schools these days. What are the tell tale signs of bad studies? And how do we or where do we find good reseached materials. Is there a clearing house?

Grover “Russ” Whitehurst:
The What Works Clearinghouse (w-w-c.org) is our effort to address the very important issue of identifying and disseminating good research with respect to questions of what works best for whom under what circumstance.


Question from Ann Brownson, Education Librarian, Eastern Illinois University:
When will the new ERIC contract be finalized? All of this concern about rigorous studies is useless if there are no ways to access the studies!

Grover “Russ” Whitehurst:
We regret the delay in the award of the new ERIC contract. The contracts office of the Department of Education is managing this process and is in the final stages of making an award.


Comment from Belinda Williams, Cognitive Psychologist, UPenn instructor:
I have read the recently developed guideline defining research based. What has been overlooked, it seems to me, is the necessary consideration of how any research fits with other relevant research e.g., interventions needing to reflect available knowledge from what we know about how the brain processes new information and how that is confirmed by available theory e.g., Piaget’s schemes, Vygotsky’s explanations of cultural dynamics in learning (valaues/interpersonal relationships, etc), Dewey’s necessary connection of curriculum with daily social knowledge, experiences and interactions.


Question from Guilbert Hentschke, Professor, University of Southern California:
In what areas of education policy and practice has research had the greatest impact, even to the point of discernably changing practice? Conversely, where has research had virtually no impact on either policy or practice?

Lisa Towne:
One of the most impressive areas is in testing and assessment, which is based on decades and decades of work in a number of fields. Although the “technology” of tests still lags behind the kinds of things we ask them to do, there have been huge advances in how we measure performance that are the core underpinnings of the tests we use today. I also think the area of reading--however controversial--has produced key insights about the ways in which kids acquire competencies in reading. The NRC summarized much of this literature in its report Preventing Reading Difficulties.

Unfortunately there are lots of areas where there hasn’t been as much impact. But it’s important to remember that this is true for a lot of reasons--including meager funding for education research and the presence of well-funded, highly ideological advocacy groups that pick and choose the research that supports their pre-determined positions.


Question from James Heinert, Superintendent, Meade School District 46-1:
While NCLB is requiring scientifically based research for interventions to support school improvement and increase student achievement, no one seems to care that NCLB itslef if not based on any research. How does one reconcile that?

Grover “Russ” Whitehurst:
NCLB is a complex, 1200 page piece of legislation. One can find within it many elements that have been influenced by research.


Question from Sally Rigeman, Consultant, Mississippi Bend AEA:
What is the status of meta-analyses in SBR?

Grover “Russ” Whitehurst:
Meta-analysis has an important role to play in synthesizing the results of research. The What Works Clearinghouse (w-w-c.org) will use meta-analytic techniques in its work.


Question from Mary Lou Ley, Educational Technology Consultant:
Even though there is a growing wealth of educational research that supports practices that improve student achievement - there is a major gap in the accessibility and implementation into classroom practice. Is there a way to build into current research proposals a stronger dissemination process or provide additional resources to teachers/schools wanting to implement such practices, like implementation grants?

Grover “Russ” Whitehurst:
You have identified a very important problem. The Institute of Education Sciences works closely with the program offices in the Department of Education to encourage not only the adoption of evidence-based programs, but also the provision of support to schools to assure that the programs are implemented with fidelity.


Question from Jean Spraker, Program Associate, Northwest Regional Educational Laboratory:
Educational research literature can be highly technical,is frequently inconclusive about causal relationships, and contains warnings about replicability beyond the context studied. People working in the implementation of k-12 education are very busy, plus they are accountable for results. What needs to happen so that quality research is availble to and used by state and local education policy and decisionmakers/leaders?

Lisa Towne:
We need a better “middle” that systematically connects the research you rightly depict and the busy lives of educators. The regional labs are part of that effort to be sure, but they’re not enough.

Another point is that research, when done properly, needs to be clear about the generalizability of its findings. As a general rule it is almost never wise to base policy or practice on a single study, no matter how rigorous or comprehensive. What educators should be demanding is high quality reviews of related literatures, and the collective recommendations of the best researchers who summarize the implications of current literature.

Finally, in the ideal research would be part and parcel of a well-functioning accountability system. Some states are already working on this, but the basic idea is that using research ought to conceived as an extension of the data-driven decision making that drives standards-based reform.


Question from Shelley Shea, Librarian, Professional Library & Archives:
Would you please expound on research that is conducted by “agenda-driven organizations” or individuals? What factors determine a possible conflict of interests in the research?

Grover “Russ” Whitehurst:
Having an agenda in the case on an organization or a strongly held hypothesis in the case of a researcher does not necessarily mean that the research is flawed. It does, however, suggest the need for careful attention by readers to the methods and conclusions that were used.


Question from Bonnie Nizamis, Consultant, The Sage Team, Inc.:
What is the statis of the What Works Clearninghouse evidence reports?

Grover “Russ” Whitehurst:
The first reports will be released this spring.


Question from Joe Paterno, Parent, Assumption High School:
Can you provide a specific example of a product or products that have been evaluated with the “gold standard” (random/control) of research advocated by the What Works Clearinghouse?

Grover “Russ” Whitehurst:
There are many randomized trials of peer assisted tutoring programs, of reading programs, of violence prevention programs, and of other products and programs that will be reviewed in due course by the WWC.


Comment from just a teacher that doesn’t want to offend anyone:
all this research is wonderful but i would like to suggest that these “researchers” and college of education experts be REQUIRED to teach ... full time ...... with a full load ....one year out of every 4 years they practice. I have seen the disparities between what these “research based” reform models claim, and what in fact they do accomplish in the real world. Teachers are bombarded with new research, and primarly from people who have not spent more than a few days in the classroom to see if it actually works, firsthand. they need to come to some of our urban schools and demonstrate through THEIR findings what works. the alchemists of long ago made many promises. we have them now giving us the same thing.


Question from Louise Wrege, preschool teacher, Benton Harbor-St. Joseph YMCA:
I have been researching the issue of school vouchers and find the research to be conflicting. For example, the first five years of the Milwaukee Parental Choice Program has been studied by three researchers who have come up with three differing opinions. How do we know who to believe? Are school vouchers a good way to raise student achievement?

Lisa Towne:
I am not an expert in this field, but my understanding of this literature is that the jury is still out on whether school vouchers are a good way to raise student achievement.

But this area of research is a good illustration of the kinds of issues we face in creating and using high quality education research. One is the concept of data-sharing. Among the three researchers you mention, there were controversies about sharing data among researchers so that results could be verified and tested under different assumptions. Data-sharing, while ensuring confidentiality and privacy, needs to be the rule rather than the exception in education research if we are to really get a handle on the kinds of important questions you raise.

The other issue that comes out of that example is that it is simply the nature of science that studies on seemingly the same topic come to different conclusions. This is as true in education research as it is in experimental ecology and number of other fields. This does not mean that the field has somehow “failed” but rather suggests that the integration of knowledge across studies--whereby scholars attempt to make sense of the findings given methodological, contextual, and other differences to try to give summary opinions on what the research says as a whole. What is needed is far more of these kinds of syntheses that help people like you make sense of studies that don’t all seem to say the same thing.


Question from Russell Mays, Associate Professor (Ed Leadership) University of North Florida:
With the very sad situation I find in many schools today, teachers not only do not feel free to experiment, they, in many cases, are forbidden from straying from prescribed curricula or methodologies. How can we expect teachers and other leaders to take the very necessary leadership role they must take in the improvement of the profession if they are bound to work with such strict limitations? I call it “educational bulimia”. How can we keep the emphasis on “scientific research” from further narrowing our focus?

Grover “Russ” Whitehurst:
Rigorous and relevant research will both narrow options by reducing use of ineffective and sometimes harmful approaches and create new options by identifying effective innovations that can be disseminated. This is true in fields such as health care and should be true in education as well. Responsible practice is either guided by the best evidence, or is part of a process of developing evidence on new practice.


Question from David Ogden, Research Associate, WestEd:
Compared to reading instruction, SBR on mathematics instruction seems to be much less well publicized. What resources do you recommend for educators regarding what works in mathematics instruction?

Lisa Towne:
The research base on mathematics education is less publicized at least in part because less is known about mathematics instruction. The NRC has a couple of publications that may be of interest--search for “Adding It Up” on our publications website (www.nap.edu). There are also companion documents to that longer synthesis of research that are targeted to practitioners.


Question from Jeannie M. Sims, Ph.D., clinical psychologist/parent:
Reviewing elementary math curricula, there seems to be more commercialism than solid research. Studies done by the sales company supporting a particular curriculum seem weak and I wonder whether they should be regarded with skepticism much like physicians bring to bear with pitches of pharmaceutical companies?

Lisa Towne:
You are right to be skeptical. Commercial publishers are trying to sell a product, and since schools are required to use products that are based on “scientifically based research”, that’s the stamp they put on them.

This is where I think the What Works Clearinghouse (Russ Whitehurst has talked about this effort quite a bit in this chat) will make the biggest difference. They will have a registry that analyzes the quality of studies of such products for people like you to access and read. That gives the kind of independent read on whether one should believe the claims of vendors.


Question from Victor Steinbok, education research and publishing consultant, Brookline, MA:
The current administration has created a climate in which pronouncements on “scientific evidence” from governmental sources are met with scepticism, if not derision. What steps is the DoEd and IES taking to insure the de-politization of the process of analyzing the research data?

Grover “Russ” Whitehurst:
All IES reports undergo rigorous peer view by external reviewers before they are published. We have established a separate office to oversee this process. As funders of research, the vast majority of the products that flow from our investments are generated and published by university-based researchers without our involvement.


Question from Mark Rohland, Staff Writer, CPRE, University of Pennsylvania:
Teachers sometimes complain that communications containing research findings are difficult to understand and use. What forms of communication--briefs, books, websites, magazines--would you suggest for presenting scientifically based research to teachers in ways that can engage them?

Lisa Towne:
This is a tough question, because I think it goes beyond issues of format. First, there’s a language issue--the dense jargon of education researchers is often quite different from the equally dense jargon of educational practitioners. So I think an important issue to address in making such “translational” products is paying close attention to the use of terms that are meaningful to to teachers. And just like effective professional development, the research findings, however packaged, need to be clearly linked to the issues and problems teachers are facing in their classrooms. Otherwise it will just be an abstraction.

Certainly brevity is important given how busy teachers are, but there is a balance to be struck here--presenting research findings to teachers can’t be so simplistic that it becomes meaningless for them; enough detail about the types of students that have been involved in the research and other such information is important to get teachers really involved in thinking through research and what it means for their practice. That’s the ultimate goal after all.


Question from Cindy Tananis, Assistant Professor, University of Pittsburgh:
Most experimental (or even quasiexperimental) research designs are focused on gathering evidence related to very specific “treatments” in controlled environments (in terms of variables and indicators of impact). The “impact” of education is designed by the nature of the “treatment” to be continuous, cumulative, sustainable and integrated. This makes for a near-impossible control of variables, context or “measurement” of traceable impact. Why then, this misguided focus on a very narrow model/description of “scientifically” based research to guide educational policy and practice? How does this serve to generate useful knowledge for practice or good information to inform policy??

Grover “Russ” Whitehurst:
There is a vast literature on the evaluation of social programs using experimental and quasi-experimental methods that does not assumed controlled environments and that provides sophisticated methods for studying variability in treatment, participation, contexts, and so on.


Question from Dr. Martin Wesche, Math Teacher, McIntosh High School, Fayette County Schools, Georgia:
Going beyond mere correlation to determining causation when comparing teaching programs usually requires random assignment of students to treatment groups. How can this best be done in the public school setting? What guidelines must be followed by educator/researchers in setting up such studies?

Grover “Russ” Whitehurst:
Although random assignment of students is sometimes necessary, the unit of random assignment often should be at higher levels such as classroom or school. Some answers to your question can be found at:

http://www.ed.gov/rschstat/eval/resources/randomqa.html


Question from David Andeson, Assessment Director, Washington Professional Educator Standards Board:
Are there other sources of systematic reviews of educational research similar to the What Works Clearinghouse?

Lisa Towne:
Yes, although not as many as I think we need. The International Campbell Collaboration will have systematic reviews in educational research and few other social science fields (e.g., criminal justice). I believe they are just starting to post their first reviews. Also, the UK has a very nice center called the Evidence for Policy and Practice Initiative (EPPI) that has many systematic reviews on the effectiveness of educational interventions, which very interesting include qualitative data.


Comment from Graduate student:
There are many ways to effectively teach reading - as long as no key components have been neglected. In Kindergarten classrooms where teachers no longer have discretion over their curriculum, you see some very uninspiring drill (on subject matter supported by scientific research). The teacher’s joy and passion related to nurturing literacy is gone. This especially happens when researchers investigated practices, but did not include teachers as partners in the development. I think scientifically researched practices should require a techer’s seal of approval.


Comment from Deanna Woods, Program Associate, Northwest Regional Educational Laboratory:
Programs like AFT’s Educational Research & Dissemination Program are designed to educate teachers themselves in reading, processing, and applying research to classroom and school practices. Wouldn’t education in general benefit from empowering teachers as professionals by making such bridge-the-gap programs more widely accessible? Don’t we want to move away from the concepts surrounding such things as teacher-proof curricula, and instead produce teachers who are professionals in the fullest sense?


Question from Deborah Meyers, Education Liaison, Partners In Brainstorms:
The DOE currently sponsors The Gateway to Educational Materials (GEM). Are we to assume that these materials have been evaluated and found to be quality research-based? And if not, how are they selected for inclusion in The Gateway? Thank you.

Grover “Russ” Whitehurst:
You should not assume that all these materials have been evaluated and are research-based. Please direct questions on GEM’s selection criteria to GEM.


Question from Paul Mills, Director, Alternative Education, Irvine Unified, California:
Significant work and research has been done here in California and throughout the nation relative to successful programs which intervene with poorly performing high school students in order to turn them around. This said, the research done relative to alternative education programs is on a particularly more remote branch of the educational research tree (as are our programs). How do we or you successfully defend our educational practices as supported by rigorous scientific research when most notable research is directed toward mainstream-based programs?

Lisa Towne:
Your question points to the issue of priorities are set for research programs. If we as a nation do not invest in the kinds of research for which there is a need, then we will have failed to make evidence-based education work right out of the box. Priority setting is a task that must be shared between researchers (who know what the current literature suggests and areas that are ripe for further inquiry) and policymakers and practitioners who need the results and who can help frame the issue.


Question from Russell Mays, Assistant Professor (Ed. Leadership) Univeristy of North Florida:
What are the top five indicators that should raise “red flags” to individuals reading research reports? (Example: research that supports a particular learning program that was conducted by the developer / seller of that program)

Grover “Russ” Whitehurst:
1) Not peer reviewed 2) Published in a journal with a very high acceptance rate 3) Published by an advocacy organization 4) Advocacy position of authors is clear within first few paragraphs 5) Research methods employed cannot in principle support the conclusions drawn in the abstract or summary


Question from MARY H. CHAMPAGNE,TEACHER,BUSINESS OWNER, CHAMPAGNE ASSOCIATES:
THE REAL PROBLEM IS THAT ADMINISTRATIONS DESPERATE TO SATISFY THE NCLB AYP ARE WILLING TO IMPLEMENT AS MANY AS FOUR COMPLETELY DIFFERENT “RESEARCH BASED PROGRAMS IN ONE SCHOOL YEAR USING BOTH STUDENTS AND FACULTY AS “LAB RATS”. LISA, PLEASE COMMENT.??????!!!!!!

Lisa Towne:
Certainly there is a tension between what we know and what resources are available and the requirements of No Child Left Behind with respect to the scientifically based research provisions. But evidence-based eduacation is not just about complying with NCLB; it’s about creating systems and instructional environments that seek out, read, and reflect on research on a regular basis as it pertains to policy and practice. I’m not minimizing what states, districts, and schools are experiencing right now given the time crunches and funding implications associated with NCLB, but I would argue that in the long term we can get these systems set up such that compliance with respect to SBR is non-sensical; it’s just a way of doing business.


Question from Patricia Pulver, professor of education, Keuka College, NY:
When teachers do research in their classrooms (action research) it is by its very nature contextual and not generalizable. How will this type of research be added to the body of knowledge? I would hate to see a system developed that does not respect the experiences and insights of classroom teachers.

Lisa Towne:
I would agree that such research ought to be a part of an evidence-based education system. The real trick is integrating what is known from more systematic research with the localized insights generated from action research as well as any number of other sources that help contextualize issues for local circumstances. Both types of research have strengths and weaknesses; the key is recognzing the role of each and figuring out ways to link them in a coherent way that can powerfully inform improvements in educational practice.


Question from Deanna Woods, retired teacher, NWREL:
What kinds of supports does IES plan to provide to ensure that teachers can master the instructional practices most effective for a diverse student body?

Grover “Russ” Whitehurst:
With the exception of the Regional Educational Laboratories, which are administered within IES, IES has no direct responsibility for providing professional development and technical assistance to teachers. Many such activities are supported by federal funds that flow through other offices of the Department.


Question from Thomas Crochunis, Research and Development, Brown University:
Given that there are many issues unresolved about how best to transact (transfer, apply, negotiate) new evidence-grounded knowledge between research and practice, are there plans for funding research into this process at the Department of Education?

Grover “Russ” Whitehurst:
We are very interested in funding research on knowledge utilization. The Interagency Education Research Initiative, a joint project of IES, NSF, and NICHD, has funded many projects that address issues of scale-up of research to practice.


Question from K. Dawn Marotta, Parent, Massachusetts:
As a parent of a child with a reading disability, I feel my head swimming as I read each new question and answer. What does scientific research mean for parents and their children? How do parents now if the program their school is using is effective, researched, and appropriate? It seems that many parents are getting the run around on what is truly researched and what is not.

Grover “Russ” Whitehurst:
I hope that the products of the What Works Clearinghouse will be useful to parents.


Comment from Russell Mays, Associate Professor (Ed leadership) University of North Florida:
YES!! I am completely in favor of “creating systems and instructional environments that seek out, read, and reflect on research on a regular basis as it pertains to policy and practice.” Getting that to the real world is the challenge. We have too many people fearing for their jobs and / or reputations and at present are giving too little attention to the real task of teaching and learning. Thank you for the insightful information this afternoon.


Question from Richard Layman, Superintendent, King & Queen County Public Schools, Virginia:
Few educational programs have been tested empirically for implementation in rural areas, especially in small, high poverty school districts located in rural areas. How does the USDE plan to assist school districts in finding programs that work in rural schools and their communities?

Grover “Russ” Whitehurst:
We have recently announced our plans to fund a research and development center on rural education. The National Center for Education Statistics, within NCES, provides a website of resources in rural education.


Question from Pier Junor, Assistant Professor, Georgia State University:
Before entering schools or districts to conduct research, isn’t there a policy where the researcher has to go through a rigorous review at their institution before conducting the actual research? If that is done, then how accountable is the researcher to the proposed study? Who monitors that process?

Lisa Towne:
Yes, the process is typically monitored by institutional review boards which are normally housed in universities. IRBs are concerned with ensuring that the rights of the research participants are upheld and that research plans include appropriate ethical safeguards.

Overall, though, successful school-based research typically depends on a partnership between researchers and practitioners. If both parties work together to identify common goals, then the real-life problems associated with doing high quality research in schools can more easily be overcome. The NRC is about to release a report next week called “Implementing Randomized Field Trials in Education” that describes these issues in more detail. Although the report talks mostly about conducting studies with randomized designs, many of the lessons are applicable to all kinds of education research done in schools. Check out www.nap.edu for the report if you are interested.


Question from Tom Kessinger, Ph.D., Assistant Professor, Xavier University:
Typically, one can read about both quantitative and qualitative research studies. Can “high-quality education research” be found in both kinds of studies? Or, is the emphasis today only on the quantitative side?

Grover “Russ” Whitehurst:
Both quantitative and qualitative research can be of high or low quality. Quality is, in part, a question of the match between the method and the question.


Question from Kim Stowe, Educational Consultant:
Many principals/teachers ask me this question...If an instructional product is based on rigourous scientific research, but the company selling it has not conducted their own research, can it be considered appropriate for NCLB?

Grover “Russ” Whitehurst:
Given a choice between an instructional product that “is based on research” and a product that has been evaluated in action, the latter is best. There are hundreds to thousands of ways to translate scientific research into the design of a particular product. Just as in a cake recipe, the proportions, time, and sequence are critical, not just the ingredients.


Debbie Viadero, Education Week (Moderator):
Those are all the questions we have time for today. I’m sorry we couldn’t get to them all. I also want to thank our guests and our audience for participating in our Talk Back Live web chat. Education Week on the Web holds these talk sessions because we think sharing information is an important way to improve education. For more information on today’s topic, you can go to the Education Department report, “Identifying and Implementing Educational Practices Supported by Rigorous Evidence: A User-Friendly Guide,” or The National Research Council report, “Scientific Inquiry in Education.”


The Fine Print
All questions are screened by an Education Week online editor and the guest speaker prior to posting. A question is not displayed until it is answered by the guest speaker. We cannot guarantee that all questions will be answered, or answered in the order of submission. Concise questions are encouraged.

Please be sure to include your name and affiliation when posting your question.

Education Week maintains Live Chat as an open forum where readers can participate in a give-and-take discussion with a variety of guests. Education Week reserves the right to condense or edit questions for clarity, but editing is kept to a minimum. Questions may also be reproduced in some form in our print edition. We attempt to correct errors in spelling, punctuation, etc. In addition, we remove statements that have the potential to be libelous or to slander someone. In cases in which people make claims that could be libelous, we will remove the names of institutions and departments. But in those cases, we will not alter the ideas contained in the questions.

Please read our Privacy Statement and Visitor Agreement if you have questions.