Student Well-Being

Reevaluating ‘Dr. Fox Effect,’ Study Finds Students Can Spot a Nonsensical Lesson

By Holly Kurtz — May 02, 2014 4 min read
  • Save to favorites
  • Print

A pair of Israeli researchers have re-opened the question of “what does the Fox say?”

In an article published earlier this year in the peer-reviewed Journal of Educational Psychology, Eyal Peer and Elisha Babad reached new conclusions when they recreated an old but famous psychological experiment known as “the Dr. Fox lecture.” Their findings contradicted the original study by suggesting that, even though students may enjoy an engaging but meaningless lecture, they are under no illusion that they have actually learned anything.

In the original study, a professional actor, introduced as “Dr. Fox,” used a humorous and engaging tone to deliver a nonsensical lecture on game theory and physician education. The three groups of psychiatrists, psychologists, social work educators, and educational administrators who were his audience subsequently gave him extremely favorable ratings. Nine even said they had read his publications.

"[S]tudent satisfaction with learning may represent little more than the illusion of having learned,” authors Donald H. Naftulin., John E. Ware, Jr., and Frank A. Donnelly concluded in the July 1973 edition of the Journal of Medical Education.

The so-called “Dr. Fox effect” has long been ammunition for critics of student evaluations of teachers. (It was even been mentioned in a 2012 Education Week opinion blog to suggest that observers can be fooled into believing that teaching is effective when it’s not.) This is despite the fact that researchers began raising serious questions about the study’s methodology and interpretation soon after it first appeared in print.

In their recreation, Peer, a senior lecturer at the Graduate School of Business Administration of Bar-Ilan University in Ramat-Gan and Elisha Babad of Hebrew University of Jerusalem, repeated the original statistical analysis, used the original questionnaire, and recreated the original setting, right down to showing the actual 1970s video footage. However, they also addressed critiques that Dr. Fox did not apply to student teacher evaluations because his audience consisted of professionals. They did so by limiting participation to university students. Unlike the original study, theirs included both treatment and control groups.

Peer and Babad also explored critiques by introducing multiple smaller variations of the original study. For instance, they offered participants opportunities to agree or disagree on two types of six-point scales rather than merely saying “yes” or “no” as they had originally been asked to do. They eliminated the original study’s introduction, in which Dr. Fox was described as a world expert in his field. They tried out the lecture on a class in which graduate students had been taught about game theory, the topic butchered by Dr. Fox.

They changed the wording of the questions from positive to negative in order to address criticisms that audience members had rated Dr. Fox favorably on the original yes/no questionnaire as a result an “acquiescence” bias in which survey respondents are more likely to agree than disagree with a statement, regardless of its content. In some conditions, they even explicitly warned students of such biases by writing: “Please pay attention. Past research demonstrated a type of bias in which questionnaire respondents tend to agree with any statement presented to them. Please try as best as you can to avoid this bias in filling out the questionnaire, and respond according to your true feelings.”

None of this made a difference. The students still rated Dr. Fox favorably.

Then, Peer and Babad introduced a new question that had not appeared on the original survey. Using a variety of formats and conditions, they asked students whether they had actually learned anything from the lecture.

Finally, they got different results. Regardless of the wording and condition, almost all of the students indicated they had not learned from the lecture. In essence, though they might have enjoyed watching Dr. Fox, they had not been duped into believing they had learned from him.

Peer and Babad suggested that future research on student evaluation of teachers should concentrate more on students’ ability to judge their own learning in various environments rather than focusing so much on their perceptions of the behaviors of their teachers.

They also questioned why the Dr. Fox study has continued to resonate in academia, despite the significant body of research critiquing it. In doing so, they mentioned a disturbing finding from another set of research studies: Namely, “80% of scientific citations had not been read by the citing authors but copied from the lists of references that appeared in other publications.”

The lead author of those citation studies, Mikhail Simkin, actually wrote to “70 of the hundreds of authors who cited the 1973 Dr. Fox article.” Of those 70, 48 responded. Just five had viewed the footage from the original study.

Here’s the kicker. In a footnote, Babad, second author of the recreation, noted that he had cited the 1973 Dr. Fox article without actually viewing the video footage. He defended himself by adding that such footage was not as accessible in the past as it is today. And he noted that he did read the article multiple times before citing it for the first time.

“However,” he concluded, “viewing the tape changed my opinion about the research and actually triggered the current research.”

You be the judge! Original footage from the Dr. Fox experiment.

A version of this news article first appeared in the Inside School Research blog.


Commenting has been disabled on edweek.org effective Sept. 8. Please visit our FAQ section for more details. To get in touch with us visit our contact page, follow us on social media, or submit a Letter to the Editor.


Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Teaching Webinar
What’s Next for Teaching and Learning? Key Trends for the New School Year
The past 18 months changed the face of education forever, leaving teachers, students, and families to adapt to unprecedented challenges in teaching and learning. As we enter the third school year affected by the pandemic—and
Content provided by Instructure
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Curriculum Webinar
How Data and Digital Curriculum Can Drive Personalized Instruction
As we return from an abnormal year, it’s an educator’s top priority to make sure the lessons learned under adversity positively impact students during the new school year. Digital curriculum has emerged from the pandemic
Content provided by Kiddom
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Equity & Diversity Webinar
Leadership for Racial Equity in Schools and Beyond
While the COVID-19 pandemic continues to reveal systemic racial disparities in educational opportunity, there are revelations to which we can and must respond. Through conscientious efforts, using an intentional focus on race, school leaders can
Content provided by Corwin

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Student Well-Being Opinion Educators, Be Future-Ready, But Don’t Ignore the Present
Being ready for what lies ahead is important, but we also need to gain a better understanding of the here and now.
5 min read
shutterstock 226918177
Shutterstock
Student Well-Being Opinion How to Prioritize Student Well-Being This Year
Use the Student Thriving Index to find out where your kids stand. Because you cannot manage what you cannot measure.
2 min read
Images shows a stylized artistic landscape with soothing colors.
Getty
Student Well-Being Spotlight Spotlight on Supporting Teachers & Students
In this Spotlight, evaluate your district and what supports your schools offer, assess attendance policies to avoid burnout, and more
Student Well-Being What the Research Says Child Hospitalizations Spike Under Delta, Particularly in Low-Vaccination States
Nationwide, the number of children and teens hospitalized due to COVID-19 has ballooned nearly tenfold since midsummer, new CDC data show.
2 min read
hopital stethescope 1222194507
Aleksandr Titov/iStock/Getty