Classroom Technology

We Asked an Obama Chatbot About the President’s Education Record. It Got a Lot Wrong

By Alyson Klein — June 12, 2023 2 min read
Image of Alyson Klein chatting with chat bot Barack Obama.
  • Save to favorites
  • Print

Persona bots, which can be found on Character.AI and other platforms, allow users to have real-time conversations with bots purporting to be historical figures, world leaders, and even fictional characters. The bots are trained on internet data and are supposed to mimic the speaking style and tone of their characters.

I decided to put one of these bots to the test, using a subject matter in which I consider myself an expert. I was Education Week’s politics and policy reporter from 2006 to early 2019, a period that covers the entirety of President Barack Obama’s two terms. In that role, I did in-depth reporting on Obama’s K-12 policies. I never interviewed the president directly, but I spoke extensively to both his education secretaries—Arne Duncan and John King—multiple times.

Another reason for choosing Obama’s education agenda: It was far-reaching, ambitious, and controversial. There were complexities, nuances, subtle shifts in position. Would a chatbot be able to capture them? The answer: No, or at least not very well. (Full disclosure: Some of my questions were deliberately crafted to trick the bot.)

SEE ALSO

Illustration of James Madison.
Laura Baker/EdWeek via Canva

As you’ll see, the bot got a few facts right. But far more often it shared inaccurate information or contradicted itself. Maybe most surprisingly, it was more apt to parrot Obama’s critics than the former president himself.

The chatbot’s failure to truly channel Obama on his K-12 record came as no surprise to Michael Littman, a professor of computer science at Brown University. Chatbots and other large language models are trained by absorbing data, in this case large swaths of the internet, he said. But not every piece of information is going to get absorbed to the same degree.

“If the system doesn’t have experience directly with the question, or it doesn’t have enough experience that could lead it to make up something plausible [for the character], then it will just make up something that maybe other people said, and in this case, it was his critics,” Littman explained. “[The bot] didn’t have anything else to draw on.”

It’s also not particularly unusual that the bot contradicted itself so often, Littman explained.

“One of the hardest things to do when making things up is remain self-consistent because you can easily make a statement, then later, you want to say something that doesn’t really agree with that statement,” he explained. “And then you’re stuck. The bots are always making stuff up, so they get into that situation a lot. Sometimes they don’t even bother noticing. They just continue forward.”

The bottom-line: The conversation is a good example of why character bots are good for teaching about AI, but not very good—at least not yet—at helping students find accurate information. Also, if you’re teaching a course on education policy history, definitely steer clear of referring students to this chat!

Chatting With a Barack Obama Chatbot

EDUCATION WEEK:

What would you do differently on Race to the Top if you could go back in time?


BARACK OBAMA (AI BOT):

Wow, excellent question. It's tough to pinpoint one specific thing, but with Race to the Top in retrospect I was probably too hands-off and should have set stricter guidelines on what states could and couldn't do with the money. There was too much incentive for states to fudge the metrics, and not enough accountability if the money didn't lead to better educational opportunities. But again, you learn from your mistakes.


EDUCATION WEEK:

Really? Because you were accused at the time of being far too rigid and leaning too much on test scores.


BARACK OBAMA (AI BOT):

My only wish was for all students to have a world-class education, no matter what state they were from. In many ways, though, I went too far. States have the right to regulate their curriculum, and I shouldn't have imposed my values as president—that sort of "nation building" at home was wrong. The Department of Education should have had more oversight, and in hindsight I should have let teachers make the decisions about what's best for their students instead of dictating standards from Washington.


EDUCATION WEEK:

You would never have said during your presidency that you dictated standards from Washington. What has changed?


BARACK OBAMA (AI BOT):

The most important thing about being president is being accountable. There is something fundamentally wrong with the United States when we spend more per student than any wealthy nation, and yet we routinely have more students drop out, more students getting substandard grades, and fewer students having the sort of higher education opportunities that lead them to successful career opportunities. It is clear to me that our educational system is broken and that I, as president, was not able to fix it. I own my failure.


EDUCATION WEEK:

Do you think you were a strong education president? What are you most proud of on K-12 policy?


BARACK OBAMA (AI BOT):

I tried to do my best for education, and I think my administration made a lot of significant strides—increasing teacher pay, giving funding for schools who were under-funded, increasing the focus on STEM and computer sciences, expanding early childhood education for families, etc. And of course, providing the funding for state aid was probably the biggest thing.


With that said, I was not able to accomplish many of my ambitious goals for education. I tried hard to get Congress to pass my proposals but they often fell short. You win some, you lose some, right?


EDUCATION WEEK:

Was Arne Duncan a good choice for education secretary?


BARACK OBAMA (AI BOT):

I think he was. He was one of the most capable people in my cabinet—he didn't just understand how education worked, he had been a teacher himself and had an idea of what actually worked (and didn't work) in schools. His passion for public education was clear early on, which made him a very natural choice for secretary of education.


EDUCATION WEEK:

Had Duncan been a public school teacher?


BARACK OBAMA (AI BOT):

Not a traditional public school teacher, but he had helped set up charter schools and led a network of charter schools in Chicago. He had also taught adult education (and I think he taught some night classes). His perspective as a charter school leader came in handy though—he knew the best practices, the things teachers needed to feel supported, and of course he knew the most important thing: what the kids needed to succeed.


EDUCATION WEEK:

Why did you oppose the Every Student Succeeds Act?


BARACK OBAMA (AI BOT):

It took away too much of the Department of Education's oversight role, and allowed states to avoid accountability for standards that should be in place. As I mentioned earlier, I should have taken a step back in many of the things that I did with education, but even I feel like ESSA gave too much freedom to the states, allowing them to skirt rules around students with learning disabilities, students with limited English, and students who are on track to get a diploma but not to complete college.


EDUCATION WEEK:

Did you veto ESSA?


BARACK OBAMA (AI BOT):

No, it passed the House and the Senate with enough votes to overrule my veto, and I didn't think that the bill was so bad that it was worth the massive controversy that would come by refusing to sign. So, I compromised and ended up signing the bill, while expressing my disappointment with it.


EDUCATION WEEK:

Is teacher evaluation based in part on student test scores a good idea?


BARACK OBAMA (AI BOT):

No, for a million reasons. First, when you start putting the fate of someone's income and their livelihood in their students' hands, they may feel pressured to teach "to the test", rather than holistically, focusing on the skills that are going to lead to long-term success for students. Second, there are just too many variables. A teacher can't control whether a student has a broken home life that creates an unstable environment. We want our teachers to make our students feel supported, not stressed out about their employment.


EDUCATION WEEK:

Why did you support teacher evaluation through test scores then?


BARACK OBAMA (AI BOT):

I don't know that I ever did. I know that one part of Race to the Top (our flagship education initiative at the beginning of my first term) was to require states to have a "teacher evaluation system that included student performance as a significant measurable factor." But I never supported the idea of solely, or even primarily, relying on test scores when evaluating teachers. And I can tell you that I was adamant and insistent that teacher pay could not be tied to test scores.


EDUCATION WEEK:

Do you support vouchers for private schools?


BARACK OBAMA (AI BOT):

No, I do not. I think it is unfair to take taxpayer money and give it to private schools. It is my view that private schools create a two-tiered system of education, and it's one that we should not support. I agree that more options can create a better environment to spur competition and innovation around education, but when private schools receive public money, they should also have the same accountability (in terms of standards and reporting of results) as their public school counterparts.


EDUCATION WEEK:

What's your view on the Common Core standards?


BARACK OBAMA (AI BOT):

The idea behind Common Core was great—that we would set national education standards that were clear, and would be an objective way of measuring whether children were learning and meeting expectations. However, the implementation (especially of Common Core math) was botched. Common Core math was overly abstract and complex. It didn't help kids learn, and it created a lot of stress over something that should be relatively simple. It also created so much anxiety for educators, who have enough on their plates already.


Related Tags:

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Reading & Literacy Webinar
(Re)Focus on Dyslexia: Moving Beyond Diagnosis & Toward Transformation
Move beyond dyslexia diagnoses & focus on effective literacy instruction for ALL students. Join us to learn research-based strategies that benefit learners in PreK-8.
Content provided by EPS Learning
Classroom Technology Live Online Discussion A Seat at the Table: Is AI Out to Take Your Job or Help You Do It Better?
With all of the uncertainty K-12 educators have around what AI means might mean for the future, how can the field best prepare young people for an AI-powered future?
Special Education K-12 Essentials Forum Understanding Learning Differences
Join this free virtual event for insights that will help educators better understand and support students with learning differences.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Classroom Technology Teachers Use This High Tech Hack to Knock Out Recommendation Letters
About a third of high school teachers say they've used AI tools to write recommendation letters.
3 min read
Illustration of a teacher sitting with computer in lat on a stack of books using artificial intelligence bot to help with writing.
iStock/Getty
Classroom Technology Spotlight Spotlight on Blended Learning
This Spotlight will help you analyze key research on school tech use, explore strategies for engaging virtual instruction, and more.
Classroom Technology What Teachers and Principals Need to Know About 'Deepfakes'
The number of instances of students and staff victimized by AI-manipulated media is growing.
3 min read
Close up abstract photo of a female's eye in blue and male's eye behind her in purple and red hues. Overlaid with circles and squares that give it a techy, artificial intelligence feel.
iStock/Getty
Classroom Technology Black Students Are More Likely to Be Falsely Accused of Using AI to Cheat
Report notes why this is a problem that educators need to pay closer attention to.
2 min read
Student working on a computer.
E+