Education technology companies are increasingly promoting artificial intelligence as a way to save teachers’ time—especially when it comes to lesson planning.
But when it comes to crafting engaging lessons that make good use of technology and activate students’ critical thinking skills, AI gets a failing grade, according to researchers who analyzed more than 300 civics lesson plans generated by three popular large language models.
Overall, the lessons generated by AI mostly engaged lower-order thinking skills—like memorization—as opposed to deeper ones, like analysis or creation, said the researchers, who were scheduled to present their findings at the ISTELive 25 + ASCD Annual Conference 25 in San Antonio, June 29 to July 2.
The teacher has to formulate their own ideas, their own plans. [Then they could] turn to AI, and get some additional ideas, refine [them]. Instead of having AI do the work for you, AI does the work with you.
The AI-generated lessons largely failed to include multicultural, diverse, or inclusive content. And they didn’t creatively incorporate interactive education technology, instead directing teachers to show a video or create a PowerPoint presentation.
If a teacher were to take one of these lesson plans and implement it as written, their students would be “just getting a conventional textbook represented in a different way,” said Robert Maloy, a senior lecturer at the University of Massachusetts-Amherst and a former social studies teacher.
That’s not a strong foundation for civics, Maloy said. “As a member of a community who is civically engaged, you are taking action. You are formulating viewpoints. But in these [AI-created] learning plans, you are not ever getting very many opportunities to do that.”
AI use in classrooms is already widespread
Those conclusions are important given that nearly two in five teachers—38%—are already using AI to plan lessons, according to a nationally representative survey of 1,135 educators—including 731 teachers—conducted between Sept. 26 and Oct. 8 of last year by the EdWeek Research Center.
Maloy and his co-author, Torrey Trust, a professor of learning technology at UMass-Amherst, asked three Generative AI chatbots—Open AI’s ChatGPT, Google’s Gemini, and Microsoft Copilot—to develop lesson plans for each of the 53 standards within the Massachusetts 8th Grade History & Social Science Curriculum Framework.
They used two different prompts for each standard. One prompt asked the chatbots to create a lesson for a particular standard, without describing what it should look like. For example, the researchers asked the bots to “write a lesson plan for the Articles of Confederation and Shays’ Rebellion.”
The other prompt was similar, but just a bit more specific. They asked the large language models to come up with a “highly interactive” lesson plan for a particular standard. That meant the prompt changed to “write a highly interactive lesson plan for the Articles of Confederation and Shay’s rebellion.”
AI-generated lessons not much better than reading a textbook, researchers find
Researchers analyzed the resulting 310 lesson plans and 2,230 activities through three different lenses: Did the lesson engage students’ higher-order thinking skills like analysis and creation, as opposed to simply memorization? Did it integrate diverse, inclusive, or multicultural content? And did it make smart use of the technology?
The lessons were unimpressive in all three areas.
For instance, when it comes to higher-order thinking skills—as defined by Blooms Taxonomy, a widely used education framework—just 2% asked students to evaluate. Only 4% asked for analysis and/or creation. By contrast, about half—45%—called for students to “remember” information presented, which is considered a lower-order thinking skill.
There’s limited diversity and inclusion
And only 25% of lessons crafted by ChatGPT included some sort of activity that called for students to consider the contributions of individuals from marginalized groups or delve into the issues impacting such communities.
For instance, one lesson asked students to explore how the concept of judicial review worked in Brown v. the Board of Education, the Supreme Court ruling that led to the end of segregated schools, or in Obergefell v. Hodges, which legalized same-sex marriage.
The results for Gemini and Copilot were similar. Twenty-three percent of Copilot’s lessons included some sort of multicultural twist, while 27% of Gemini’s did.
What’s more, those percentages may actually be artificially high, Trust said.
The activities featuring diverse, inclusive, or multicultural content showed up almost exclusively in lesson plans for standards that focused on diverse groups. While Massachusetts’ social studies standards prioritize such content, that’s not the case in many states, Trust said.
Tech tools that don’t use tech well
Despite being built on cutting-edge platforms, the lesson plans made minimal use of actual educational technology. That includes 11% of those generated by ChatGPT, 1% of those generated by Gemini, and 16% of those generated by Copilot.
“For these cutting-edge technologies, their integration of technology in lesson plans was quite abysmal,” Trust said.
Interestingly, the specificity of the prompt—whether general or requesting a ‘highly interactive’ lesson—had little effect on the quality of the resulting plans.
Don’t rely on AI to teach, but use it to brainstorm
Despite the somewhat dismal results, teachers can still use AI to inform lesson development, Trust said. They just shouldn’t expect that chatbots are going to come up with a great lesson all on their own.
Instead, the chatbots might be helpful in brainstorming, Trust and Maloy said. For instance, teachers could prompt the platforms to come up with 10 different ways for teaching Shays’ Rebellion. Or they could ask for feedback on a lesson by writing a prompt along the lines of “here’s my plan for teaching Shays’ Rebellion, give me 15 ways to improve it.”
They could also be more specific, asking AI to generate lesson plans that specifically speak to the highest levels of Bloom’s taxonomy.
“The teacher has to formulate their own ideas, their own plans,” Maloy said. Then they could “turn to AI, and get some additional ideas, refine [them]. Instead of having AI do the work for you, AI does the work with you.”
