Using ChatGPT to grade essays and give detailed feedback: For Teachers

Summary

In this thought-provoking video, a teacher named Mr. Storm explores the potential of AI language models like ChatGPT in grading written assignments. He tests the system's ability to evaluate and provide feedback on three essays (excellent, average, and poor) based on a standard rubric. Surprisingly, ChatGPT not only grades the essays accurately but also offers detailed, constructive feedback tailored to each essay's strengths and weaknesses. The teacher is amazed by the AI's ability to provide kind, diplomatic, and encouraging feedback, which he admits is something he often struggles with due to time constraints. This experience inspires him to reconsider his approach and strive to provide more detailed, supportive feedback to his students.

Takeaways

• 😲 The teacher was initially nervous about using ChatGPT to grade written assignments, unsure if it would work effectively.
• 📝 Three sample essays were used: a great award-winning essay, a middle-level essay with some errors, and a poor-quality essay with significant issues.
• 🤖 ChatGPT was able to successfully grade the essays based on a provided rubric, assigning appropriate grades and providing feedback.
• 💯 For the great essay, ChatGPT recognized its clarity, focus, and effective presentation of the writer's perspective on racism.
• 🅲 For the middle-level essay, ChatGPT identified issues such as lack of detailed support, weak introduction and conclusion, and limited word choice and grammar errors.
• 🅓 For the poor essay, ChatGPT noted its unfocused topic, lack of organization, and numerous grammar, mechanics, and spelling errors.
• 📝 ChatGPT provided constructive and encouraging feedback for the poor essay, highlighting areas for improvement while maintaining a positive tone.
• 🤩 The teacher was impressed by ChatGPT's ability to provide detailed, personalized feedback that encourages students to improve their writing skills.
• ⏳ The teacher acknowledged that providing such feedback manually would be time-consuming, but ChatGPT made it more feasible.
• 🌟 The experience motivated the teacher to want to provide better feedback to students and inspired them to potentially rewrite the poor essay based on ChatGPT's suggestions.

Q & A

What was the motivation behind testing ChatGPT's ability to grade written assignments?

The motivation was to explore the potential of using ChatGPT as a tool for grading written assignments, particularly essays. The person in the video (Mr. Storm) was curious to see if ChatGPT could effectively evaluate and provide feedback on essays based on a standard rubric.

What types of essays were used for testing ChatGPT's grading capabilities?

Three different types of essays were used: (1) an award-winning essay (considered a 'great' essay), (2) a mid-level essay with some grammar errors and basic quality (predicted to be a low B or high C grade), and (3) a poorly written essay (predicted to be a D grade).

How did ChatGPT perform in grading the 'great' essay?

ChatGPT was able to effectively evaluate the 'great' essay and provide positive feedback, recognizing its clear focus, well-supported ideas, strong organization, and the writer's extensive knowledge on the topic.

How did ChatGPT handle grading the mid-level essay?

For the mid-level essay, ChatGPT identified its clear topic but noted that the main ideas were not well-supported by detailed information. It also pointed out issues with the introduction, lack of a conclusion, limited word choice, and grammatical/spelling errors that interfered with understanding.

What was remarkable about ChatGPT's feedback on the poorly written essay?

Despite the essay being of poor quality, ChatGPT provided constructive and encouraging feedback. It acknowledged the writer's passion but also identified areas for improvement, such as focus, organization, grammar, and spelling, while maintaining a positive and motivating tone.

What impressed Mr. Storm the most about ChatGPT's performance?

Mr. Storm was impressed by ChatGPT's ability to provide detailed and personalized feedback while maintaining a kind and diplomatic approach. He felt that the feedback was more encouraging and supportive than what he typically provides to students due to time constraints.

How did ChatGPT's feedback compare to Mr. Storm's usual approach?

Mr. Storm acknowledged that he often struggles to provide detailed feedback to students due to time constraints. However, ChatGPT's feedback was not only thorough but also kind and encouraging, motivating students to improve without feeling attacked or discouraged.

What was Mr. Storm's overall reaction to the experience?

Mr. Storm was blown away by ChatGPT's performance and felt encouraged by the potential of using such a tool to provide high-quality feedback to students. He expressed feeling bad for not providing similar levels of feedback in the past and was motivated to explore incorporating ChatGPT or similar tools in his teaching practice.

What did Mr. Storm suggest would be a typical workflow for using ChatGPT's feedback?

Mr. Storm suggested that he could use ChatGPT's detailed feedback as the basis for an email to the student, adding a short personal statement and offering his availability for further guidance. This would save time while still providing thorough and personalized feedback.

What broader implications did this experience have for Mr. Storm?

The experience with ChatGPT made Mr. Storm realize the potential of AI tools in education, specifically for providing high-quality feedback to students. It encouraged him to explore incorporating such tools in his teaching practice and potentially rethinking traditional approaches to feedback and grading.

Outlines

00:00

🤖 Using ChatGPT to Grade Student Essays

The paragraph describes the narrator's idea of using ChatGPT to grade student essays. They provide three sample essays of varying quality (great, mediocre, and bad) and an essay rubric. The narrator asks ChatGPT to grade the great essay using the rubric, and ChatGPT provides positive feedback, identifying the essay as well-written and matching the rubric criteria. The narrator is impressed with ChatGPT's ability to grade and provide constructive feedback.

05:01

🙌 Impressed by ChatGPT's Diplomatic Feedback

The narrator is amazed by ChatGPT's ability to provide kind and diplomatic feedback on the mediocre and bad essays, highlighting areas for improvement while maintaining a positive and encouraging tone. ChatGPT also offers suggestions on how to improve the bad essay, with detailed feedback that would typically take the narrator a significant amount of time to write. The narrator is motivated to provide better feedback to students and is grateful for ChatGPT's belief in their potential to succeed.

Keywords

💡Essay Grading

The main focus of the video is exploring the potential of using ChatGPT to grade written essays. The speaker expresses both curiosity and apprehension about this idea, seeing it as a way to streamline the grading process but also questioning its effectiveness. Several sample essays of varying quality are provided to test ChatGPT's ability to evaluate them based on a standard rubric.

💡Rubric

A rubric is a set of criteria used for evaluating or grading assignments, projects, or performances. In the video, the speaker refers to a 'basic essay rubric' found online, which outlines the expectations and standards for different components of an essay, such as focus, organization, and mechanics. The rubric serves as a guide for ChatGPT to assess and score the provided essay examples.

💡Feedback

Feedback is the information or critique provided to a student or learner regarding their work or performance. The speaker is impressed by ChatGPT's ability to generate detailed, constructive feedback for each essay sample, highlighting strengths and weaknesses while offering suggestions for improvement. The feedback is described as kind, diplomatic, and encouraging, qualities that the speaker finds valuable for motivating students.

💡Time-saving

One of the potential benefits of using ChatGPT for essay grading that the speaker acknowledges is the time-saving aspect. He mentions that providing thorough, personalized feedback to students can be time-consuming, and ChatGPT's ability to generate detailed feedback quickly could be a useful tool for teachers with limited time.

💡Constructive Criticism

The speaker is impressed by ChatGPT's capacity for constructive criticism, which involves providing feedback in a positive and supportive manner while still identifying areas for improvement. The feedback generated for the 'bad essay' sample is described as kind and encouraging, motivating the hypothetical student to try again and improve their writing skills.

💡Personalized Learning

The detailed feedback provided by ChatGPT for each essay sample aligns with the concept of personalized learning, which involves tailoring educational experiences to individual students' needs, strengths, and weaknesses. The speaker recognizes the value of such personalized guidance in helping students develop their writing abilities at their own pace.

💡Artificial Intelligence

Although not explicitly stated, the video explores the potential applications of artificial intelligence (AI) in education, specifically in the context of essay grading and feedback. ChatGPT, as an AI language model, demonstrates capabilities that the speaker finds impressive and potentially transformative for the teaching and learning process.

💡Student Engagement

The speaker expresses his desire to provide the kind of detailed, encouraging feedback that ChatGPT generates, recognizing its potential to enhance student engagement and motivation. By offering constructive guidance in a supportive manner, students may be more likely to remain engaged and invested in improving their writing skills.

💡Writing Skills

The core skill being assessed and developed in the video is writing ability, specifically in the context of essay writing. The speaker provides essay samples of varying quality to test ChatGPT's capacity to evaluate and provide feedback on different aspects of writing, such as focus, organization, mechanics, and language use.

💡Educational Technology

The video explores the integration of educational technology, in this case, the use of an AI language model like ChatGPT, into the teaching and learning process. The speaker is intrigued by the potential of leveraging such technology to streamline grading, provide personalized feedback, and enhance the overall learning experience for students.

Highlights

Mr. Storm explores ChatGPT's ability to grade essays against a basic rubric.

Three essays are tested: an award-winning essay, a mid-level essay, and a poor-quality essay.

The award-winning essay receives high praise for its clarity, organization, and insight on racism.

The mid-level essay is graded a C, with feedback on lack of detail and conclusion.

The poor-quality essay is graded a D, criticized for its unfocused topic and numerous errors.

ChatGPT provides constructive feedback for the poor-quality essay, focusing on its strong passion but suggesting improvements.

Feedback emphasizes the importance of organization, clarity, and avoiding grammar errors.

ChatGPT offers detailed suggestions for improvement, encouraging specificity and focus.

Mr. Storm is impressed by ChatGPT's ability to provide kind, diplomatic, and detailed feedback.

The process demonstrates ChatGPT's potential to save teachers time in grading and providing feedback.

ChatGPT's feedback is seen as encouraging, helping students see their potential for improvement.

Mr. Storm acknowledges the challenge of providing detailed feedback due to time constraints.

The experiment leads to a realization of the importance of detailed feedback for student improvement.

ChatGPT is viewed as a tool that believes in students' potential and encourages them to succeed.

Mr. Storm concludes the video encouraged by ChatGPT's capabilities in educational feedback.

The video ends with an invitation for others to try ChatGPT for educational purposes.

Transcripts

hey everybody this is Mr storm and um I

got the idea that maybe I

maybe I could see if chat GPT can grade

written assignments

um

I'm kind of nervous about this one

because I'm not quite sure if I want

this to work

um

but let's give it a shot so I found this

basic essay rubric online for you know a

standard rubric for writing an essay

um I'm gonna go ahead and copy

actually I'll leave that alone for now I

have three examples of essays I have a

pretty great essay this is an

award-winning essay

um this is a mid-level essay just a

standard basic I just went on Google and

said hey give me a basic standard High

School personal essay

um there's some

some grammar errors here some stuff like

that I read through the essay it's

pretty middling I'd probably give it

maybe a low B high C Maybe

and then we have a bad essay uh this one

is just not great

um probably a d for this one depending

on the student depending on you know

other stuff using words like grody

um you know it's just not great

um so I figured I'd kind of check chat

gpt's ability to determine

um how these essays match up with this

rubric

so grade this essay

uh let's try the the great one first

let's copy and paste hello

um we'll just see if it can grade it

let's say is well this essay is well

written and presents personal

perspective on the experience of racism

the writer effectively conveys the

impact of racism in their daily life in

various forms it can take the micro from

micro address aggressions in the

classroom blah blah blah this is great

so that would be nice feedback that you

can give to the student let's try to

match it up with this rubric and see if

we can make this work

uh now grade that essay based on this

rubric

and the formatting is weird but we'll

see if it can make sense of it

oh my gosh

the essay has a clear well-focused topic

and the main ideas are well supported by

detailed and accurate information so

focus in details

um the introduction is inviting and

provides an overview of the paper and

the conclusion is strong

so organization

the essay demonstrates a writer's

extensive knowledge of the experience so

I could

ah that is great

um so that works so

congratulations on your a paper

let's try this mid-level essay to see

what we come up with

I'm gonna go ahead and copy and paste

um

use oh hello

use that rubric to grade this paper

a c that was my prediction

the essay has a clear topic but the main

ideas are not well supported by detailed

information the introduction States the

main topic and provides an overview of

the paper but there is no conclusion

the essay lacks evidence of the writer's

extensive knowledge or experience with

the topic and the word choice is limited

with a Reliance on jargon and cliches

the sentence structure is well

constructed but the grammar and

mechanics contain several errors that

interfere with understanding there's

also a spelling error in the assay

okay well that's kind of what I

predicted let's see how well it does

with the bad essay

a d

the essay has a clear but unfocused

topic and the main ideas were not well

supported by the detailed information

there's no introduction or conclusion

and the essay lacks evidence of the

writer's extensive knowledge or

experience with the topic the word

choice is limited and the writing

contains numerous errors and grammar

mechanics and spelling that interfere

with understanding the essay is also

disorganized and frequently veers off

topic

um wow uh right

let's see

write feedback

for that student

um

but make it

nice

oh my God

it's clear that you have a strong

passion for chocolate and the many

different ways it can be enjoyed while

your enthusiasm is appreciated it would

be helpful to focus your writing on a

specific topic and provide more detailed

information to support your ideas paying

attention to grammar mechanics and

spelling can also help your writing to

be more clear and easy to understand

keep up the good work and try to focus

on organizing your thoughts and

presenting them in a clear and concise

manner you guys that is some pretty good

feedback I mean you're giving them

exactly

what they need to hear and they can see

on the rubric what they've missed but

you're but but it's positive

I don't feel like I was just attacked I

feel like maybe I'm on the right track

if I try again I might do better

how is this thing better

at being

kind

and diplomatic then a lot of people I

know to be honest with you this is

insane

um give me some suggestions on how to

make that paper

better

make it detailed oh I'd spell detailed

wrong

here if you okay

choose a specific focused topic the

essay currently touches on a number of

different this is detailed feedback on

how to make it better for next time

think about how long this would take for

you to write

all of this

really critical feedback for your

students

personally

would take quite a while like literally

I could take

I could use this

to grade on the rubric

I could pop this onto the top of an

email

pop this into the bottom of an email

write a short statement

encouraging

a student to try again

and let them know I'm available to help

it's great to see that you're trying to

improve your writing skills don't get

discouraged if you don't see immediate

Improvement writing is a skill that

takes time to practice and develop

remember that it's okay to make mistakes

and that asking for help is normal and

important part of the learning process

if you ever need help or have questions

I'm here to support you and provide

guidance keep working hard and don't be

afraid to try again I know you have the

potential to succeed guys I want to

rewrite this paper for that teacher this

teacher makes me want to try again

this chatbot Believes In Me you guys

and it wants me to be better

this is the kind of feedback we should

be giving our students

and and honestly I'll just be honest

with you the reason I don't give this

kind of detailed feedback is just

because I don't have time

I know what I want to say I just don't

have time to generate the information

but this

guys

what is going on

I mean

this is fantastic I'm okay I'm gonna cut

this video here just because I mean I'm

kind of Blown Away with how good this is

I I did not expect this to work as well

as it did

and I'm starting to kind of like feel

bad for not giving this kind of feedback

to my students previously

I'm kind of encouraged by this honestly

um all right well

definitely

I don't know give this a shot and let me

know what you come up with

all right thanks for uh paying attention

and I'll I'll see you next time

bye