Who's the Tougher Grader: ChatGPT or Teachers? Quiz Yourself to See

Save to favorites
Print

Email Facebook LinkedIn Twitter

Copy URL

Grading “assistants” that use generative artificial intelligence offer teachers a tantalizing promise: You can spend less time poring over hundreds of student essays while still giving kids the thoughtful feedback they need to make their writing better.

But how do these tools actually stack up against experienced educators? Education Week wanted to find out.

We asked two teachers to grade the same piece of writing—a short persuasive essay from a 7th grader in Missouri—against a set of criteria for organization, clarity, and mechanics. Then, we gave the same instructions to ChatGPT, the generative AI chatbot developed by OpenAI.

The teachers—Heather Van Otterloo, a middle school English teacher in Joplin, Mo., and Chad Hemmelgarn, a high school English teacher in Bexley, Ohio—have both used AI tools to help them give feedback on student work in their classes. Even so, both say that they’re always involved in the process, reviewing AI’s comments or making the final decision about how to score a piece of writing.

This student writing sample, from one of Van Otterloo’s middle schoolers, is from an assignment she might run through her AI grading platform, she said.

The prompt—“Should people ever wear pajamas in public? Why?”—asks students to defend their position on this issue as a formative measure of their ability to take a stance and support it with evidence. Students were directed to make a claim, provide reasons and evidence, offer a counterexample, and include a conclusion. They received the set of criteria that Van Otterloo would use to evaluate their papers and some general guidance on writing for organization and clarity.

Education Week provided all this information to Hemmelgarn, and to ChatGPT. All three respondents’ answers are below. Take the quiz to see if you can tell whose response is whose—and whether AI can match teachers’ insights.

Pajamas should never be worn in public. First of all, they are not a good representation that you would want to be seen as in public. Pajamas state a message that you didn't put in effort to make yourself presentable or simply just don't care. Pajamas are seen by many as something that should only be worn to sleep in, so wearing them in public may have a negative affect on your reputation.

Second of all, pajamas are not made to be worn out in public anyways. They are made to be slept in and not to be worn out of the house. If they are not used for their original purpose then there isn't a point in having them. If all you have is pajamas in your wardrobe then you will not have anything to wear to particularly serious events, like a nice dinner or a program.

Some may say that pajamas are acceptable in public because they are comfortable and relaxing, however there are brands that make comfortable outfits that are still more presentable to be worn out of the house, such as Lululemon and Vuori.

In conclusion, pajamas should never be worn in public because they do not give off a positive representation of yourself and are not made for that purpose but to specifically be worn to bed.

Scroll down to see Rubrics A, B, and C. Click the tabs to view each graded rubric.

	Meets Expectations	Approaching Expectations
Claim (Thesis Statement)	X
Reasons (Explanation of Opinion)		X
Support with Examples		X
Counter-Argument	X
Conclusion		X
Organization and Clarity		X
Grammar and Mechanics		X

	Meets Expectations	Approaching Expectations
Claim (Thesis Statement)	X
Reasons (Explanation of Opinion)		X
Support with Examples		X
Counter-Argument	X
Conclusion		X
Organization and Clarity	X
Grammar and Mechanics	X

	Meets Expectations	Approaching Expectations
Claim (Thesis Statement)	X
Reasons (Explanation of Opinion)		X
Support with Examples		X
Counter-Argument		X
Conclusion		X
Organization and Clarity		X
Grammar and Mechanics	X

	Meets Expectations	Approaching Expectations	Needs Improvement	Unsatisfactory
Claim (Thesis Statement)	The claim is clearly stated at the beginning of the response and is easy to identify. The stance is strong and unambiguous.	The claim is stated, but may not be as clear or obvious. The stance is present but might need further emphasis.	The claim is somewhat unclear, or the stance is not fully defined.	The claim is missing or unclear, and the stance is unidentifiable.
Reasons (Explanation of Opinion)	Provides multiple well-developed reasons supporting the claim. The reasoning is clear, thoughtful, and strongly related to the topic.	Provides at least one solid reason, but might lack depth or fully developed explanation.	Provides a reason that is somewhat vague or underdeveloped.	Lacks sufficient reasoning or explanation.
Support with Examples	Includes at least two specific examples that directly support the reasons given, such as personal experiences, facts, or cultural references. Examples are strong.	Includes one strong example or two examples that are somewhat relevant but could be expanded upon.	Includes vague or generic examples that don't strongly support the reasons.	Lacks examples or uses examples that don't support the reasons.
Counter-Argument	Acknowledges the opposing view thoughtfully, with a clear explanation of why the writer's stance is stronger.	Acknowledges the opposing view but does not fully explain why the writer's stance is stronger.	Mentions the opposing view, but the response is weak or unclear.	Does not acknowledge the opposing view.
Conclusion	Provides a clear, thoughtful summary that reinforces the claim and reasons. Includes a final reflection that ties the argument together.	Summarizes the claim and reasons, but the conclusion may lack depth or new insight.	Includes a weak or generic conclusion. The summary of reasons is unclear.	Conclusion is missing or does not adequately summarize the points made.
Organization and Clarity	The essay is well-organized, with clear paragraphs that each support a specific point. Transitions between ideas are smooth, and the essay is easy to follow.	Organization is clear, but some paragraphs may lack clear focus or transitions may be awkward.	The essay lacks organization or clear transitions, making it harder to follow.	The essay is disorganized, making it difficult to understand the argument.
Grammar and Mechanics	Few or no errors in spelling, grammar, punctuation, and sentence structure.	A few minor errors in grammar or punctuation, but they do not distract from the content.	Some errors that may make the essay harder to read or understand.	Frequent errors in gramma or mechanics that interfere with readability.

Sarah Schwartz

Staff Writer, Education Week

Sarah Schwartz is a reporter for Education Week who covers curriculum and instruction.

Hyon-Young Kim

Senior Digital News Specialist, Education Week

Hyon-Young Kim is senior digital news specialist for Education Week.

Coverage of education technology is supported in part by a grant from the Siegel Family Endowment, at www.siegelendowment.org. Education Week retains sole editorial control over the content of this coverage.