ASU professor talks use of AI, ChatGPT in universities and its future

Summary

The video discusses the impact of AI on university students, especially with tools like ChatGPT. The professor explains how AI has evolved and how it is used in academia. While AI helps students with tasks like improving writing, it also raises concerns about plagiarism and misinformation. The professor highlights the ongoing cat-and-mouse game between educators and students adapting to AI's capabilities. They mention that despite challenges, there's potential for positive use in education. The discussion touches on the need for institutions to develop strategies to handle AI's influence in academic settings.

Takeaways

• 💡 AI is fundamentally reshaping how students approach research, learning, and exams, offering both new opportunities and challenges.
• 🖥️ The professor explains that AI's recent surge in impact is due to the rise of internet data, smartphones, and machine learning algorithms, making tasks like image and text generation more accessible.
• 📚 Students can now use AI tools like ChatGPT to assist with writing and research, but this raises concerns about plagiarism and academic integrity.
• 🔍 AI systems like ChatGPT don't retrieve exact passages but rather generate plausible completions of prompts, often introducing factual inaccuracies known as 'hallucinations.'
• 🎓 Academia is grappling with how to respond to AI's influence, with some universities considering banning these tools while others explore how to integrate them responsibly into education.
• 📝 The rise of AI writing tools has blurred the lines between authentic student work and machine-generated content, leading professors to rethink plagiarism detection and writing assessments.
• 🌍 The professor highlights AI's widespread influence, noting that ASU students are actively using these tools, and its impact is visible in Silicon Valley and beyond.
• ⚠️ Large language models (LLMs) can sometimes produce misinformation, fabricating sources or generating fictional stories, creating new challenges for verifying facts in both academic and legal contexts.
• 🧠 A watermarking system could help identify AI-generated content by manipulating word choices in generated text, but its adoption depends on cooperation from AI developers.
• 🔮 Despite concerns, the professor believes society will adapt to AI's presence, drawing parallels to how people learned to spot AI-generated essays in academic contexts over time.

Q & A

What is the primary focus of the professor's department?

The department focuses on computer science and artificial intelligence, particularly on technologies that assist humans in performing intelligent tasks. It also covers general computer science topics like hardware, software, and software engineering.

How does the school collaborate with the outside world?

The school collaborates with various companies and government agencies, often through sponsored research. The professor's research, for example, is funded by federal agencies like the Department of Defense (DOD) and the National Science Foundation (NSF). Students also contribute by interning at industry partners, particularly in Silicon Valley.

What significant changes have occurred in the field of AI over the past few decades?

AI has evolved from performing tasks that humans understand, like playing chess, to tackling complex tasks like image and text recognition. The recent availability of large datasets from the internet has allowed AI to learn tacit knowledge that humans find difficult to articulate, like recognizing images or generating text.

How is AI impacting students' ability to write papers and take tests?

AI tools like ChatGPT can assist students by improving their writing, especially for those who are non-native English speakers. However, there is also concern about plagiarism, as AI-generated essays can be difficult to detect since they paraphrase rather than directly copying text.

What challenges do large language models (LLMs) like ChatGPT pose in academic settings?

LLMs present challenges in detecting plagiarism since they generate original-looking content by paraphrasing rather than copying. They also have issues with 'hallucinations,' where they fabricate information, making it harder for educators to trust the output.

How can AI tools improve student writing, and what concerns exist?

AI tools can help students refine their rough ideas, making their writing clearer and more coherent. However, there is concern that students may rely on AI to do their work for them, especially in areas like plagiarism and over-reliance on AI-generated content.

What proactive steps are universities taking to address the challenges posed by AI?

Some universities are restricting access to AI tools, while others are exploring ways to use AI proactively, such as asking students to fact-check or correct AI-generated content. There is also ongoing research into watermarking AI-generated text to identify its source.

How might future tools help detect AI-generated content in academic work?

One idea is to watermark AI-generated content by using a technique that prevents certain words from being used. By knowing which words were blacklisted during generation, professors could potentially detect AI-generated work, as human writing wouldn’t follow the same constraints.

How are students currently using AI tools like ChatGPT?

Students are using AI to paraphrase texts, generate summaries, and assist with research. However, there are cases where AI tools fabricate citations or other information, which can lead to academic issues.

What are some of the limitations of AI tools like ChatGPT in generating accurate information?

ChatGPT and similar tools often generate the 'most likely' completion for a given prompt, which can result in the creation of false information, such as fabricated citations or inaccurate descriptions. This makes it unreliable for fact-based tasks.

Outlines

00:00

🧠 Introduction to AI in Education

The speaker begins by introducing the topic of AI's impact on university students, particularly as they approach finals. They interview a professor from the school of computing and AI, who explains the department's focus on technologies that aid humans in intelligent tasks. The professor also discusses the department's research and its collaboration with federal agencies and industry partners. The conversation highlights the rapid advancements in AI and how they have been integrated into the curriculum and student life.

05:00

📚 AI's Impact on Student Research and Plagiarism

The discussion shifts to how AI has made research easier for students but also facilitated plagiarism. The professor explains that tools like Turnitin are used to detect copied work, but AI technologies like chatbots can generate original text that is difficult to trace. This has led to a new challenge in academia, as educators must now discern whether student work is original or AI-assisted. The professor also notes the positive use of AI for improving writing skills, especially for non-native English speakers.

10:03

🎓 The Evolution of AI in Academia

The professor reflects on the evolution of AI, noting its origins in the 1950s and the significant advancements in the last 10-15 years due to the availability of large datasets on the internet. He compares AI learning to human learning, where systems now can generate text and complete tasks in ways that were previously challenging. The conversation also touches on how AI has changed the landscape of student assignments and exams, making it easier to generate text but also raising concerns about the authenticity of student work.

15:03

📝 Combating Plagiarism with AI

The discussion delves into the strategies professors are using to combat AI-assisted plagiarism. The professor suggests that students should verify the information generated by AI systems and correct any inaccuracies. He also mentions the idea of watermarking AI-generated text to identify its origin. The conversation highlights the need for students to critically evaluate AI-generated content and the ethical implications of using AI in academic work.

20:04

🔮 The Future of AI in Education

The professor speculates on the future of AI in education, suggesting that educators and students will adapt to the new challenges posed by AI. He shares anecdotes about how quickly professors have learned to identify AI-generated essays and predicts that the cat-and-mouse game between educators and students will continue to evolve. The conversation concludes with the professor's thoughts on the potential for AI to both enhance and complicate the educational experience.

Keywords

💡Artificial Intelligence (AI)

Artificial Intelligence refers to the development of computer systems that can perform tasks typically requiring human intelligence. In the video, AI is presented as a field of study with roots going back to 1956 and has seen significant growth in recent years, particularly with technologies that allow machines to perform tasks that are difficult to explain in human terms, such as painting or generating text.

💡Large Language Models (LLMs)

Large Language Models are a type of AI that can generate human-like text by predicting word sequences. The video explains how technologies like ChatGPT, which is an example of an LLM, are changing how students write papers and communicate. These models complete text in a manner similar to how humans recall information, often paraphrasing or 'hallucinating' details.

💡Plagiarism

Plagiarism is the act of using someone else's work or ideas without proper attribution. In the context of the video, AI tools like ChatGPT complicate plagiarism detection because the generated text is not directly copied from a specific source, making it difficult for plagiarism detection tools to identify as plagiarized work.

💡ChatGPT

ChatGPT is a prominent AI tool discussed in the video that generates text based on prompts. Its widespread use among students has raised concerns about academic integrity, as it can produce coherent essays that aren't directly copied but might still constitute a form of plagiarism. ChatGPT can also introduce factual errors, further complicating its use in academic settings.

💡Hallucination

In the context of AI, hallucination refers to the phenomenon where AI models like ChatGPT generate information that appears factual but is actually made up. The video discusses how LLMs can invent convincing details, such as fake papers or biographies, which can mislead users into believing they are accurate.

💡Turnitin

Turnitin is a plagiarism detection tool mentioned in the video that compares student submissions to known text and code repositories. It was originally developed to catch direct copying, but with the advent of AI-generated text, Turnitin faces new challenges since tools like ChatGPT do not directly retrieve and copy content.

💡Paraphrasing

Paraphrasing involves rewording text while maintaining its original meaning. The video highlights how students use AI to paraphrase or rewrite essays, making it harder for traditional plagiarism detection tools to catch these instances since the AI-generated text does not match exact sources.

💡Watermarking

Watermarking in the context of AI-generated text refers to embedding identifiable markers to indicate that the text was produced by an AI. The video explores how watermarking could be a future solution to detecting AI-generated essays, helping educators differentiate between human-written and AI-produced work.

💡Silicon Valley

Silicon Valley is referenced in the video as a major tech hub where many students from the professor’s university work after graduation. It symbolizes the close relationship between academic institutions and industry, particularly in fields like computer science and AI, where students contribute to cutting-edge technology development.

💡Explicit vs. Tacit Knowledge

Explicit knowledge refers to information that can be clearly communicated, such as the rules of chess, while tacit knowledge is harder to articulate, like how to swim or walk. The video contrasts AI's early focus on tasks with explicit knowledge (e.g., chess) to more recent advancements in tacit knowledge tasks (e.g., image recognition, language generation).

Highlights

AI's impact on university students as they approach finals.

Professor's name and title in the School of Computing and AI.

Department's focus on technologies that assist humans in intelligent tasks.

Research sponsored by federal agencies and industry partners.

Students' significant presence in Silicon Valley and their impact.

AI's evolution since 1956 and the shift from explicit to tacit knowledge.

The role of internet and mobile technology in AI advancement.

Challenges in detecting plagiarism with AI-generated text.

Use of AI to improve writing skills, especially for non-native English speakers.

Academic concerns over AI-generated content and plagiarism since the release of ChatGPT.

Strategies to identify AI-generated content, such as checking for factual accuracy.

The potential for AI to 'hallucinate' and create false information.

The idea of watermarking AI-generated text to detect its origin.

Students as early adopters of AI technology for various purposes.

The use of blacklisted words to identify AI-generated text.

The commercial applications of AI beyond academia.

The adaptability of educators and students to AI-generated content.

The future of AI in education and the potential for new detection methods.

Transcripts

well yeah so it's like it's become very

you okay okay well thanks for talking to

us if it's okay just an overview we we

kind of started this wondering

how much AI may or may not be changing

the landscape of students at

universities as we head to finals so

that's kind of how we started this thing

if it's okay I'll ask you a little bit

um first first give me your your name

first and last in the title please okay

um I am

and I'm a professor in the school of

computing and AI

um spell that Professor for me so I get

it right first and last name

yeah so what what happens in your

department here what kind of things do

you do this this is the Department of

computer science and artificial

intelligence augmented intelligence and

so basically we are interested in

looking at technologies that

um help humans in you know whatever

considered intelligent tasks in addition

to General computer science topics such

as you know Hardware software software

engineering and so on so that's the

sorts of things that we have that happen

and there's a bunch of work and research

going on and in all sorts of AI

artificial intelligence topics um in in

the department in the school

yeah be careful with your hands you're

bouncing it okay oh yeah okay okay how

does this school and what you do here in

end up translating to the outside world

you team up with companies or government

agencies yes so lots of that so most of

the research is sponsored by you know

the federal agencies in my own case you

know multiple federal agencies DOD

reports NSF and

and also lots of us get

gifts or the grants from more like

contracts from local industry or you

know National Industry partners and so

that's one main way of course you know

as a educational institution our biggest

way we actually help the industry and

the world in general is our students so

our students go on internships um so in

fact it's you can't go to Silicon Valley

and throw a stone without hitting any

you know Sunday well essentially I mean

so computer science ASU computer science

students are all over Silicon Valley so

so that's those are like the main

impacts that we have

tell me about

um

how fast and how far we've come with

something like AI because to us out in

the outside world it seems like

overnight boom it was there

um from your point of view so yeah I

think as a field AI has been around

since 1956 actually I think that's you

know there was like this meeting when

they acquired the name and I personally

have been working since my undergrad day

so close to 40 years

um the main thing that changed is in the

beginning we were getting computers to

do what are intelligent tasks that we do

but we know how we do them things like

playing chess we not only play chess we

have the rules of Chess and we know how

to play chess we can tell people how to

play chess

um but there are many other things we do

such as walking such as swimming such as

singing such as painting that we do

that's extremely hard to explain in in

words those sorts of things we basically

could not get machines to do in the last

10 15 years basically with the Advent of

web um you know internet and in a cell

phone technology Etc this is like a tons

and tons of data of on about all these

activities that's on the web and so you

can actually train systems using that

data in a way that's not that different

from what happens to our kids you know

the kids basically in the beginning

learn to walk learn to you know in the

swim or whatever basically by trial and

error and by looking at the world and

it's only the things like math that we

teach them later so the funnier thing is

that kids went that way from you know

tacit what's called asset knowledge to

explicit knowledge AI systems did

explicit knowledge tasks before like you

know in back in late 90s we already

defeated casparov in in chess but now we

can actually recognize movies recognize

pictures and develop you know generate

pictures you know corresponding to any

prompt and most recently generate text

corresponding to any prompt and that

essentially is all of writing so pretty

much all of conversation all of writing

is I ask you to you know tell me

something about the following topic and

you start talking and then I interrupt

and say something else and then you

continue and so so it becomes easier to

generate these sorts of things now how

is this sir changed the landscape of

what students are doing to write their

papers and take tests and do finals

I I think so you know technology has

been you know evolving quite a bit over

the last many years and so when Google

came along already uh students can do a

lot more research a lot easier without

having to actually step out and go to

the library they can essentially sit in

front of the computer and look at

Wikipedia look at this and all these

things that was the positive side the

negative side is that if they didn't

really want to do their own thinking

they can copy from Wikipedia copy from

various books and submit those things

and this became kind of a cat and mouse

game and so Technologies are developed

on the educational side which can spot

directly copied passages right like

there are there are companies like

turnitin and various other things which

will compare people's essay students

essays or even students code to known

code repositories and known tax

repositories they can get cut what

changed now so that is already going on

right now and what changed now with the

Advent of these sorts of chat GPT style

Technologies

is they don't directly index and

retrieve entire passages it's sort of

like your and my memory if we read about

something and we are telling it to our

friend we give the gist of it in our own

words mostly because we don't actually

remember the exact passage that we read

we are just essentially able to complete

you know the gist these systems almost

work that way ah essentially they are

stringing together words uh that would

be a logical completion of the prompt or

the question that you gave them what

that essentially means is that the essay

they put together for example will never

be anywhere on the Wikipedia you can't

prove that the students have actually

copied it verbatim from anywhere else

um and so that obviously is a new kink

in this cat and mouse came again

everything has a huge positive side for

example

students can use these sorts of systems

to improve their writing you know they

can put together their rough thoughts in

maybe English that's somewhat rough

especially when you have lots and lots

of students who for whom English is not

you know not necessarily a first

language and then they can have their

chat GPT style llms large language

models to rewrite the text in fact I can

tell you that I get tons of mails from

International students asking for

internships asking for researchers and

positions Etc and ever since chat GPT

the quality of English has become more

like Midwest now so it's no longer

doesn't sound like there is no such

thing as Indian English or Chinese

English it's all whatever chat gbt

rights sort of a thing that could be

seen as a positive thing because they

are getting their message across but in

terms that I can understand but the

other side of it is they can also use it

to for things like plagiarism

are professors around here you can

Target professors worried about that

around here I yeah in in general

Academia has been quite worried since

like December because I think November

30th is when open AI put Chachi PT out

into complete open

um usage and and the university started

hyperventilating as to how especially

certain kinds of things like SCA based

evaluation you know in English and so on

it becomes a much harder thing and and

so the universities are actually having

including it ASU they are having you

know

um think tanks and discussion groups to

figure out what is the best way to

handle this you you know some have taken

extreme steps in some universities like

we don't allow access to chat GPT

um but others are taking actually a more

interesting proactive steps for example

one of the things that's well known is

as I said charging PPT and are are llms

the large language models and I keep

saying GPT but there are many other

things like Bard in Google and so on all

of these are essentially dynamically

generating the answer and it's not the

answer is not really factual are are

intentionally a lie it is just a most

likely completion of what the question

was so if you said this question it's

almost like if I say he you know stepped

onto the road and looked left end and

then you will complete in your mind

right because that's the most likely

completion except they're doing this you

know with much bigger context you know

given the last 3000 words what would be

the most likely words that we need to

string together

when they do that they can actually make

up stuff you know it's been called

hallucination but essentially everything

that is we do this human memory is very

valuable which is why in courts when

Witnesses are not trusted like right

away because we our memory was almost

this way we string together pieces to

make into stories sometimes the stories

are not actually completely true and and

these systems have the same exact

problem and so some professors have

actually used this to say okay you can

ask chargpt for the answer to this

factual question but then you need to

correct it

do you see what I'm saying because it's

like it it sounds it sounds very

authentic because that's part of our

problem we tend to think that people who

speak smoothly also know truth right

that part is like a cognitive flaw that

you know human we have we humans have

that we think that people who are well

dressed people who speak

smoothly must know what they're doing

that's why most of the Khan jobs are by

smooth trackers not like people with

right but that has to change now because

everybody can speak smoothly with you

know charging in the background and so

from the professor side we need to

actually look at the content which is

longer it takes more time and from the

student side you can ask them if you use

chat GPT if it's a factual question it's

a question which actually has you know

answers that are true versus false you

need to go through what it said and

figure out what is right and what is

wrong to give you an example if I go and

ask GPT 4 which is like the latest and

greatest system give me a 500 word bio

of Subaru Karma party that's myself

right you know because that's the most

narcissistic thing to do and it will

give a 500 word bio

it would look more or less kind of

correct and it turns out that it makes

up a whole bunch of things but I would

know that

do you see what I'm saying so for

example you know when at the last I

asked it it said I'm a president of a

particular Professional Organization I

was a president of triple AI which is

you know artificial intelligence

organization but it said I was a

president of computational linguistics

Organization for somebody outside they

can't tell you know it looks you know

maybe it's right do you want someone to

say but I do so the point is so if

suppose somebody were to make a

biography for some you know question I

can make them check double check with

Google so still by the way it turns out

that Google when it points you to

um credible sources like Wikipedia is

still a lot more factual than chargpt

and these llms can be and currently

that's just the reality they are

basically like you know worse than our

memories and our memories are valuable

you know we can remember a few things

but if you want me if I ask you exactly

what happened yesterday in your life you

would basically unintentionally

non-develop greatly make up a whole

bunch of stuff you know it's not

necessarily that you're trying to lie

it's just because you haven't indexed it

it's not a video in your head you are

putting pieces together the order will

be mixed up you may have done one

activity before other activity but

you'll remember the other way around

that's exactly what these systems wind

up doing

um and and so you can use that you know

basically with the students and say if

you're using it you know double check

that there are errors there are other

ways in which you can actually catch

plagiarism in terms of so one idea would

be if you use you know a large language

model to produce some text can there be

a watermark on it so that I can tell

that this part was produced by these

kinds of systems this significant

research that's going on there right now

and one particularly interesting idea is

essentially because these systems have

this ability that suppose if I were to

ask you to tell me your life story

without using the following 15 words

it's hard for us in fact I think you

know there are like TV games of this

kind saying you know say whatever you

want to say but without using Volume 15

words you can make charity PPT to write

an essay without using the following

twenty thousand words

so you know English approximately has 50

000 word functional vocabulary so I can

say don't use this Blacklist words right

right and then the the thing that comes

out it's paraphrased and so you can't

tell the difference it will be still

English okay but then if you know what

the Blacklist words are then you know

whether in fact it was generated by the

large language model are normal people

because normal people don't know what

the Blacklist is so they would actually

start using those words this is a this

is actually a pretty good way of

checking a watermarking the text except

for this to work the people selling

these systems need to want to help you

know and it's not very clear that they

want to because if they're selling this

service they probably would want people

to able to be able to say we wrote this

because the bigger Money Maker is the

large group of students around not just

students it's actually it's I mean we

are talking because we are in the

University we are talking about the

students but really this stuff was not

made for students I mean students are

the smartest ones they are always ahead

of the car when they try to use the

technology for themselves but this is

for anybody this is for writers this is

for liars you know every day you read

that Liars instead of writing like an

entire case summary they tend to write

some you know bulleted Point stuff um

and then chat GPT are these large

language models converted into English

in fact there's a joke now that you know

people will take five bullet points that

they want to send an email to their

friends and give it to charge GPT it

will make it into a two-page letter very

nicely Written Letter and then those

friends will ask Chachi PPT to convert

into five bullet points right so between

us we are just increasing that you know

that traffic on the internet but in in a

weird sense you know we have this

problem you know as humans we want

things to be well written but we also

want gist of it in bullet points you

know if you anyway wanted bullet points

why can't I just send you bullet points

you will be offended if somebody writes

you a letter with just bullet points but

it wants you need to have this so you

know Allah is full of this and so

basically they can have this this sort

of uses of you know these kinds of

Technologies and even there and that

could be a reasonable use

um right in the case of students of

course you might be more interested in

plagiarism detection but you know and

and so this Blacklist whitelist thoughts

of approaches our way are a way of you

know capturing that a couple more things

that I'm done

um you're right yeah

how much do you think uh students are

just using this technology in general or

is it still new enough I think again I

think typically the students I'm in the

College of Engineering our students are

of course the smartest in being the

early adopters of All Tech in general

you know I mean I'm sure I mean we all

know that our kids are faster with

technology

um than than you know the older people

right and so so people are already using

this and they're using it for all sorts

of uses one of the most interesting

things is these systems were generated

as text completing systems that's all so

like you know it was there for a long

time for five years back onwards as

you're typing on your iPhone sometimes

it car you know basically suggests the

word because it assumes that this is the

word you are likely to type instead of

suggesting the word and the spelling now

they can suggest sequence of words

that's essentially the how the way the

technology will develop but people found

that it can actually be used you know

given that all of conversation all of

you know language activity is saying

something I say something and you say

something that is connected to that and

I say something that is connected to

what you said and so you can use llms

you can use this to do that and so

students have essentially actually one

of the early ones who have figured out

that you can use these sorts of

technique systems

to kind of paraphrase things to ask

instead of going to Google and ask tell

me all the research papers in this area

and give me a summary of what happened

in this area previously they had to go

old old days they had to go to the

library more recently they had to click

on Google Now they just asked chat GPT

it will give a summary but when it gives

a summary oftentimes it's not necessary

that it's actually true it just the

summary is the most likely sequence of

words sometimes it's actually correct

sometimes it's not they've been cases

where charging Beauty would generate

citations that is actual paper names

with the names of journals they were

published in all made up these journals

don't exist these papers don't actually

exist so I actually did the same thing I

did one more narcissism question where I

asked charging Beauty for you know in an

area I work in human every eye systems

what are some of the best papers hoping

that you'll say you know one of my

papers it did say one of my papers

except I never wrote that paper

so I quickly actually have to write one

and put it on archive so that people

will cite it so the point again is

students are finding very interesting

ways they have actually been cases where

recently in the news for example people

have found based on what Chachi PPT said

people got into trouble like it made up

an entirely convincing story that's a

mayor was a criminal and that poor guy

wasn't or that a law professor actually

harassed his students sexually harassed

students and there was no such thing if

you ask a leading question it gives a

nice answer to it and it has its

completely there's this old movie called

absence of malice and that's playing out

in in screen right now because chat GPT

doesn't mean to hurt you mean to help

you it is just trying to complete the

sentence and all the meaning and all the

truth Etc is in your head when you

actually look at it and so students

are finding the you know both the uses

and also kind of finding the pitfalls of

it and and I believe that it's it's a

it's an interesting new world right now

yeah you can make it cat Matt Mouse game

cat and mouse gas will move on exactly

to the next level yeah what do you think

is is next coming um down the pike for

all of this type of thing I I think this

type of knowledge my senses will get

used to it in fact one of the most

interesting thing hopeful things that

I've heard is this philosophy Professor

I mean after all philosophy you would

think philosophy I say should be easy to

make right um and this philosophy

Professor basically had like two things

to say when in back in December when the

charging came he said by January he was

like trying bucket saying suddenly all

my students essays looks so much more

coherent and I know that they clearly

are not writing this stuff okay by March

he was saying that I can spot a chat GPT

essay from a mile away

do you understand what I'm saying

because we get used to it I mean it's

like

we do always we can still tell the

difference between people who are faking

it and people who are actually who know

what they are talking about maybe you

can be taken a couple of days a couple

of times but if you are actually

interacting with the person who is

Faking it mostly without knowing what

they're doing you