Computerized Essay-Scoring Helpful for Student Feedback?

By Benjamin Herold — August 21, 2014 2 min read
  • Save to favorites
  • Print

This article by Liana Helton is cross-posted from the Curriculum Matters blog.

Despite the many negative associations with automated essay-scoring, there’s some evidence it may actually be more effective in changing student behavior than human scoring, Annie Murphy Paul wrote recently in the Hechinger Report.

For the most part, discussions about computerized grading (a.k.a. robo-scoring, machine scoring, A.I. scoring) center on how it’s used in assessment. Both PARCC and Smarter Balanced, the two state consortia developing common-core-aligned tests for next spring, considered using artificial intelligence for scoring writing tasks, but are likely to go with hand scorers for now.

The typical back-and-forth on the topic (and there’s plenty of it) is about accuracy. A 2012 study found there were no significant differences between human and computer scorers. But can computerized scoring systems truly gauge a student’s grasp of language and writing skills? Can the systems be cheated, as Les Perelman, a former director of writing at the Massachusetts Institute of Technology, suggests? Are they able to measure creativity?

Paul, in the Hechinger Report, comes at this from a different angle. She suggests computerized scoring can be a valuable teaching tool because students react better to receiving edits from a computer than from a human.

Citing a 2010 study by Khaled El Ebyary, a professor of applied linguistics at Alexandria University in Egypt, and Scott Windeatt, a senior lecturer in the Center for Research in Linguistics and Language Science at Newcastle University in England, she writes that, when interacting with the computerized editor, students were much more likely to have positive feelings and to revise their work. Some students even saw it as game-like, which boosted their motivation. Meanwhile, “comments and criticism from a human instructor actually had a negative effect on students’ attitudes about revision and on their willingness to write,” according to Paul.

Students experience a “disinhibition effect” with the technology, she said—similar to the one that makes medical patients more likely to answer health questions truthfully when on a computer than when sitting face-to-face with a practitioner.

“It’s the very non-humanness of a computer that may encourage students to experiment, to explore, to share a messy rough draft without self-consciousness or embarrassment,” Paul writes. “In return, they get feedback that is individualized, but not personal--not ‘punitive.’”

Earlier this year, Education Week contributing writer Caralee Adams wrote that the technology for automated scoring is becoming more sophisticated, and that more players are entering the market. She quotes a 7th grade English teacher whose students use an online essay-scoring program to get feedback before they hand in their work.

“Is it perfect? No,” he said. “But when I reach that 67th essay, I’m not real accurate, either. As a team, we are pretty good.”

Follow @BenjaminBHerold and @EdWeekEdTech for the latest news on ed-tech policies, practices, and trends.

A version of this news article first appeared in the Digital Education blog.

Commenting has been disabled on effective Sept. 8. Please visit our FAQ section for more details. To get in touch with us visit our contact page, follow us on social media, or submit a Letter to the Editor.


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Teaching Webinar
6 Key Trends in Teaching and Learning
As we enter the third school year affected by the pandemic—and a return to the classroom for many—we come better prepared, but questions remain. How will the last year impact teaching and learning this school
Content provided by Instructure
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Equity & Diversity Webinar
Leadership for Racial Equity in Schools and Beyond
While the COVID-19 pandemic continues to reveal systemic racial disparities in educational opportunity, there are revelations to which we can and must respond. Through conscientious efforts, using an intentional focus on race, school leaders can
Content provided by Corwin
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Equity & Diversity Webinar
Evaluating Equity to Drive District-Wide Action this School Year
Educational leaders are charged with ensuring all students receive equitable access to a high-quality education. Yet equity is more than an action. It is a lens through which we continuously review instructional practices and student
Content provided by BetterLesson

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment Spotlight Spotlight on Assessment in 2021
In this Spotlight, review newest assessment scores, see how districts will catch up with their supports for disabled students, plus more.
Assessment 'Nation's Report Card' Has a New Reading Framework, After a Drawn-Out Battle Over Equity
The new framework for the National Assessment of Educational Progress will guide development of the 2026 reading test.
10 min read
results 925693186 02
Assessment Opinion Q&A Collections: Assessment
Scores of educators share commentaries on the use of assessments in schools.
5 min read
Images shows colorful speech bubbles that say "Q," "&," and "A."
Assessment Standardized Tests Could Be in Jeopardy in Wake of Biden Decisions, Experts Say
Has the Biden administration shored up statewide tests this year only to risk undermining long-term public backing for them?
6 min read
Image of a test sheet.