Harvard Professor Explains Algorithms in 5 Levels of Difficulty | WIRED

Summary

视频脚本通过一系列访谈和示例，探讨了算法在日常生活中的普遍应用和重要性。从计算机科学教授到数据科学家，不同领域的专家分享了他们对算法的理解，包括搜索算法、机器学习和深度学习等。同时，讨论了算法的未来发展，以及它们如何影响我们的生活和工作。视频强调了算法的基础知识和不断进步的技术之间的联系，以及对这些技术潜在的利弊进行批判性思考的重要性。

Takeaways

• 📚 算法无处不在，无论是在物理世界还是虚拟世界中都扮演着重要角色。
• 👨‍🏫 哈佛大学计算机科学教授David J. Malan强调算法是解决问题的机会。
• 🖥️ 计算机的核心部件包括CPU（中央处理器）和内存（RAM）。
• 🔍 算法可以是有步骤的指令集，用于指导人们或机器人完成特定任务。
• 🥪 通过制作三明治的例子，展示了算法需要精确的指令来正确执行。
• 🔎 搜索算法如Google和Bing背后的技术是快速有效地查找信息的关键。
• 📈 数据科学和机器学习领域中，算法用于优化模型和解决业务问题。
• 🤖 机器人学习结合了机器人技术和机器学习，用于开发智能系统。
• 🧠 人工智能和机器学习的发展，使得算法在日常生活中的应用越来越广泛。
• 🚀 算法的研究和发明涉及思考效率和连接不同领域的知识。
• 🌐 算法的进步和理解与我们对技术的掌握和推进是松散耦合的。
• 💡 尽管大型语言模型等先进技术的发展，基础算法的学习仍然对于理解技术至关重要。

Q & A

算法的重要性体现在哪些方面？

算法的重要性体现在它们无处不在，无论是在物理世界还是虚拟世界中。算法为解决问题提供了机会，而我们生活中处处都有问题需要解决。

计算机科学教授是如何定义计算机的？

计算机是电子设备，形状像矩形，可以进行敲击输入操作。

CPU和内存（RAM）在计算机中分别扮演什么角色？

CPU是计算机的大脑，负责响应指令，进行数学运算等操作。内存（RAM）则用于存储正在使用的程序和数据。

硬盘（HDD或SSD）与内存（RAM）有什么区别？

内存（RAM）是易失性存储，用于存储当前正在使用的程序和数据。而硬盘（HDD或SSD）是非易失性存储，用于永久保存数据、文档等信息，即使断电也能保持信息不丢失。

算法可以如何帮助我们提高效率？

算法通过提供精确、逐步的指令来解决问题，从而提高效率。例如，在搜索算法中，通过优化搜索过程，可以更快地找到所需信息。

什么是递归算法？

递归算法是一种算法，它通过重复使用自身来解决问题，每次解决的都是问题的一个更小的部分，直到问题被完全解决。

在数据科学领域，算法的作用是什么？

在数据科学领域，算法主要用于优化模型，找到最佳的数据集描述，或者作为数据产品的核心，帮助解决新闻编辑室和商业问题。

大型语言模型（LLM）在人工智能中扮演什么角色？

大型语言模型是人工智能的一个特殊领域，主要用于预测下一个词或一系列符号。它们可以用于生成文本、聊天机器人等应用，并且在不断学习和优化以提高性能。

为什么说我们对算法的理解与我们的进步是松散耦合的？

我们对算法的理解与我们的进步是松散耦合的，因为即使我们不理解算法的底层原理，也可以使用高级工具和库来实现复杂的功能。然而，深入理解算法仍然对于开发和优化更高效的算法至关重要。

算法在日常生活中的应用有哪些？

算法在日常生活中的应用非常广泛，包括搜索引擎优化、推荐系统（如YouTube、Netflix）、社交媒体内容推荐（如TikTok的For You页面）等。算法帮助我们更有效地找到和组织信息，提高我们的生活质量。

对于想要进入计算机科学领域的学生或成人，大型语言模型（LLM）的出现意味着什么？

大型语言模型（LLM）的出现并不意味着我们应该避免进入计算机科学领域。相反，它们提供了新的学习和研究机会，让我们可以探索如何训练、微调和应用这些模型来解决实际问题。

Outlines

00:00

📚 算法的基础与重要性

本段落介绍了计算机科学教授David J. Malan对算法的解释，强调了算法无处不在的重要性，无论是在物理世界还是虚拟世界。他通过日常生活中的例子，如制作三明治的步骤，解释了算法的概念，并指出算法的核心在于解决问题。此外，他还讨论了计算机的两个关键硬件组件：CPU和内存，并解释了它们的作用。

05:00

🔍 搜索算法的效率与优化

这一部分讨论了搜索算法的效率问题，通过查找电话簿中的联系人为例，说明了不同搜索算法的效率差异。首先提到了线性搜索的低效率，然后介绍了分而治之的策略，即每次跳过一定数量的条目来快速缩小搜索范围。这种方法比线性搜索快得多，因为它通过不断减半搜索范围来加速查找过程。

10:01

📈 算法的递归性与复杂性

在这一部分中，讨论了更复杂的算法类型——递归算法。递归算法是一种自我调用的算法，通过将问题分解成更小的部分来解决问题。通过与嘉宾Patricia的对话，解释了在计算机科学和数据科学中算法的应用，并通过磁性数字在黑板上的排序示例，展示了如何使用简单的步骤来解决看似复杂的问题。

15:01

🤖 机器学习与算法研究

本段落探讨了机器学习和算法研究的关系。作为第四年的博士生，讨论者分享了他在机器人学习和机器学习方面的研究经验。他强调了算法研究的重要性，尤其是在寻找低效问题和连接不同领域之间的联系方面。此外，讨论了机器学习在推荐系统、搜索引擎和社交媒体中的应用，以及算法如何影响我们的日常生活。

20:02

🌐 算法在数据科学和产业中的角色

在这一部分中，Chris Wiggins教授讨论了算法在数据科学和产业中的应用。他解释了数据科学家如何使用优化算法来找到最佳的数据集模型，并如何将算法转化为数据产品。此外，他还提到了算法在新闻室和商业问题中的应用，以及数据科学家如何将算法集成到他们的工作中。最后，他讨论了人工智能的兴起，以及它如何改变人们对算法的看法。

25:04

🚀 算法的未来发展与影响

最后一部分讨论了算法未来发展的可能方向，以及它们对我们生活的影响。讨论者提出了算法将在我们日常生活中扮演越来越重要的角色，并且它们可能会在一些意想不到的地方发挥作用。同时，他们也讨论了大数据和机器学习算法可能带来的风险，以及如何平衡算法的益处和潜在的负面影响。

Keywords

💡算法

算法是一系列定义清晰的指令序列，用于解决特定问题或执行特定任务。在视频中，算法被描述为无处不在的工具，无论是在物理世界还是虚拟世界中。算法的重要性在于它们提供了解决问题的机会，并且与我们日常生活中的许多活动息息相关。例如，视频中提到了搜索算法、排序算法和机器学习算法等。

💡计算机

计算机是一种电子设备，能够接收和处理数据，执行各种计算和任务。它通常由中央处理单元（CPU）和内存（RAM）等硬件组成，这些部件负责解释和执行指令。在视频中，计算机被比喻为一个矩形的设备，用户可以通过键盘输入指令来与之交互。

💡中央处理单元（CPU）

中央处理单元，简称CPU，是计算机的核心硬件之一，负责解释和执行计算机程序中的指令。CPU是计算机的大脑，能够处理各种计算任务，如数学运算和逻辑判断。

💡随机访问存储器（RAM）

随机访问存储器，简称RAM，是计算机的另一种重要硬件，用于临时存储正在运行的程序和当前使用的数据。与CPU协同工作，RAM确保计算机能够快速访问和处理信息。

💡精确性

精确性是指在执行任务或解决问题时的准确性和正确性。在算法和计算机程序中，精确性尤为重要，因为即使是微小的错误也可能导致完全不同的结果。

💡分治法

分治法是一种算法设计策略，它通过将大问题分解为小问题，然后递归地解决这些小问题来简化问题解决过程。这种方法通常用于提高解决问题的效率。

💡递归算法

递归算法是一种特殊的算法，它通过重复调用自身来解决问题。这种方法通常用于解决可以自然分解为更小、更简单子问题的问题。

💡机器学习

机器学习是人工智能的一个分支，它使计算机系统能够通过经验自我改进。这通常涉及使用大量数据训练模型，以便模型能够识别模式并做出预测或决策。

💡数据科学

数据科学是一门综合性学科，它使用统计学、机器学习和软件工程技术来分析、处理和解释大量数据，以发现有价值的信息和支持决策。

💡优化算法

优化算法是一类用于在给定的约束条件下寻找最佳解决方案的算法。这些算法在数据科学中尤为重要，因为它们可以帮助找到最适合数据集的模型或描述。

💡人工智能

人工智能是指使计算机系统模拟人类智能行为的技术，包括学习、推理、自我改进、感知和理解语言等能力。

Highlights

哈佛大学计算机科学教授David J. Malan解释算法的重要性及其在现实世界和虚拟世界中的普遍应用。

算法是解决问题的机会，生活中每个人都会遇到问题需要解决。

计算机被形象地描述为可以像手机一样电子化操作的矩形设备，能够响应指令进行计算。

介绍了计算机的核心硬件CPU（中央处理单元）和内存（RAM）的概念。

算法被定义为一系列指令，用于指导人们或机器人行动。

举例说明了日常生活中的算法，如睡前例行程序和制作三明治的步骤。

精确性在算法指令中的重要性，以及它在现实世界问题解决中的应用。

通过与人的互动，展示了如何创建一个简单的算法来解决特定问题。

讨论了搜索算法的概念，以及如何在电话簿中查找联系人的示例。

介绍了分治算法的概念，以及它如何通过将问题分解成更小的部分来提高解决问题的效率。

递归算法的定义，以及它如何通过自我复制来解决相同的问题。

讨论了算法在社交媒体和搜索引擎中的应用，以及它们如何影响我们日常生活的各个方面。

纽约大学的博士生讨论了算法研究和发明的过程，以及机器学习在算法发展中的作用。

哥伦比亚大学的副教授和纽约时报的首席数据科学家Chris Wiggins讨论了数据科学中的算法应用。

探讨了人工智能和数据科学之间的联系，以及它们如何共同推动技术进步。

讨论了大型语言模型（LLMs）和Chat GPT等工具对计算机科学领域的影响，以及它们如何改变人们对AI的看法。

强调了即使在高级算法看似遥不可及的情况下，通过学习和研究，最终也能够理解和掌握它们。

Transcripts

- Hello world.

My name is David J. Malan

and I'm a professor of computer science

at Harvard University.

Today, I've been asked to explain algorithms

in five levels of increasing difficulty.

Algorithms are important

because they really are everywhere,

not only in the physical world,

but certainly in the virtual world as well.

And in fact, what excites me about algorithms

is that they really represent an opportunity

to solve problems.

And I dare say, no matter what you do in life,

all of us have problems to solve.

So, I'm a computer science professor,

so I spend a lot of time with computers.

How would you define a computer for them?

- Well, a computer is electronic,

like a phone but it's a rectangle,

and you can type like tick, tick, tick.

And you work on it.

- Nice. Do you know any of the parts

that are inside of a computer?

- No.

- Can I explain a couple of them to you?

- Yeah.

- So, inside of every computer is some kind of brain

and the technical term for that is CPU,

or central processing unit.

And those are the pieces of hardware

that know how to respond to those instructions.

Like moving up or down, or left or right,

knows how to do math like addition and subtraction.

And then there's at least one other type of

hardware inside of a computer called memory

or RAM, if you've heard of this?

- I know of memory because you have to memorize stuff.

- Yeah, exactly.

And computers have even different types of memory.

They have what's called RAM, random access memory,

which is where your games, where your programs

are stored while they're being used.

But then it also has a hard drive,

or a solid state drive, which is where your data,

your high scores, your documents,

once you start writing essays and stories in the future.

- It stays there.

- Stays permanently.

So, even if the power goes out,

the computer can still remember that information.

- It's still there because

the computer can't just like delete all of the words itself.

- Hopefully not.

- Because your fingers could only do that.

Like you have to use your finger to delete

all of the stuff. - Exactly.

- You have to write.

- Yeah, have you heard of an algorithm before?

- Yes. Algorithm is a list of instructions to tell people

what to do or like a robot what to do.

- Yeah, exactly.

It's just step by step instructions for doing something,

for solving a problem, for instance.

- Yeah, so like if you have a bedtime routine,

then at first you say, "I get dressed, I brush my teeth,

I read a little story, and then I go to bed."

- All right.

Well how about another algorithm?

Like what do you tend to eat for lunch?

Any types of sandwiches you like?

- I eat peanut butter.

- Let me get some supplies from the cupboard here.

So, should we make an algorithm together?

- Yeah.

- Why don't we do it this way?

Why don't we pretend like I'm a computer

or maybe I'm a robot, so I only understand your instructions

and so I want you to feed me, no pun intended, an algorithm.

So, step-by-step instructions for solving this problem.

But remember, algorithms, you have to be precise,

you have to give...

- The right instructions.

- [David] The right instructions.

Just do it for me. So, step one was what?

- Open the bag.

- [David] Okay. Opening the bag of bread.

- Stop. - [David] Now what?

- Grab the bread and put it on the plate.

- [David] Grab the bread and put it on the plate.

- Take all the bread back and put it back in there.

[David laughing]

- So, that's like an undo command.

- Yeah.

- Little control Z? Okay.

- Take one bread and put it on the plate.

- Okay.

- Take the lid off the peanut butter.

- [David] Okay, take the lid off the peanut butter.

- Put the lid down.

- [David] Okay. - Take the knife.

- [David] Take the knife.

- [Addison] Put the blade inside the peanut butter

and spread the peanut butter on the bread.

- I'm going to take out some peanut butter

and I'm going to spread the peanut butter on the bread.

- I put a lot of peanut butter on

because I love peanut butter.

- Oh, apparently. I thought I was messing with you here...

- No, no it's fine.

But I think you're apparently happy with this.

- [Addison] Put the knife down,

and then grab one bread and put it on top

of the second bread, sideways.

- Sideways.

- Like put it flat on.

- Oh, flat ways, okay.

- [Addison] And now, done. You're done with your sandwich.

- Should we take a delicious bite?

- Yep. Let's take a bite.

- [David] Okay, here we go.

What would the next step be here?

- Clean all this mess up.

[David laughing]

- Clean all this mess up, right.

We made an algorithm, step by step instructions

for solving some problem.

And if you think about now,

how we made peanut butter and jelly sandwiches,

sometimes we were imprecise and you didn't give me

quite enough information to do the algorithm correctly,

and that's why I took out so much bread.

Precision, being very, very correct with your instructions

is so important in the real world

because for instance, when you're using the worldwide web

and you're searching for something on Google or Bing...

- You want to do the right thing.

- [David] Exactly.

- So, like if you type in just Google,

then you won't find the answer to your question.

- Pretty much everything we do in life is an algorithm,

even if we don't use that fancy word to describe it.

Because you and I are sort of following instructions

either that we came up with ourselves

or maybe our parents told us how to do these things.

And so, those are just algorithms.

But when you start using algorithms in computers,

that's when you start writing code.

[upbeat music]

What do you know about algorithms?

- Nothing really, at all honestly.

I think it's just probably a way to store information

in computers.

- And I dare say, even though you might not have

put this word on it, odds are you executed as a human,

multiple algorithms today even before you came here today.

Like what were a few things that you did?

- I got ready.

- Okay. And get ready. What does that mean?

- Brushing my teeth, brushing my hair.

- [David] Okay.

- Getting dressed.

- Okay, so all of those, frankly, if we really

dove more deeply, could be broken down into

step-by-step instructions.

And presumably your mom, your dad, someone in the past

sort of programmed you as a human to know what to do.

And then after that, as a smart human,

you can sort of take it from there

and you don't need their help anymore.

But that's kind of what we're doing

when we program computers.

Something maybe even more familiar nowadays,

like odds are you have a cell phone.

Your contacts or your address book.

But let me ask you why that is.

Like why does Apple or Google or anyone else

bother alphabetizing your contacts?

- I just assumed it would be easier to navigate.

- What if your friend happened to be at the very bottom

of this randomly organized list?

Why is that a problem? Like he or she's still there.

- I guess it would take a while to get to

while you're scrolling.

- That, in of itself, is kind of a problem

or it's an inefficient solution to the problem.

So, it turns out that back in my day,

before there were cell phones, everyone's numbers

from my schools were literally printed in a book,

and everyone in my town and my city, my state

was printed in an actual phone book.

Even if you've never seen this technology before,

how would you propose verbally to find John

in this phone book? - Or I would just flip through

and just look for the J's I guess.

- Yeah. So, let me propose that we start that way.

I could just start at the beginning

and step by step I could just look at each page,

looking for John, looking for John.

Now even if you've never seen this here technology before,

it turns out this is exactly what your phone could be doing

in software, like someone from Google or Apple or the like,

they could write software that uses a technique

in programming known as a loop,

and a loop, as the word implies,

is just sort of do something again and again.

What if instead of starting from the beginning

and going one page at a time,

what if I, or what if your phone goes like two pages

or two names at a time?

Would this be correct do you think?

- Well you could skip over John, I think.

- In what sense?

- If he's in one of the middle pages that you skipped over.

- Yeah, so sort of accidentally and frankly

with like 50/50 probability,

John could get sandwiched in between two pages.

But does that mean I have to throw

that algorithm out altogether?

- Maybe you could use that strategy until you get close

to the section and then switch to going one by one.

- Okay, that's nice.

So, you could kind of like go twice as fast

but then kind of pump the brakes as you near your exit

on the highway, or in this case near the J section

of the book.

- Exactly.

- And maybe alternatively, if I get to like

A, B, C, D, E, F, G, H, I, J, K,

if I get to the K section,

then I could just double back like one page

just to make sure John didn't get sandwiched

between those pages.

So, the nice thing about that second algorithm

is that I'm flying through the phone book

like two pages at a time.

So, 2, 4, 6, 8, 10, 12.

It's not perfect, it's not necessarily correct

but it is if I just take one extra step.

So, I think it's fixable,

but what your phone is probably doing

and frankly what I and like my parents and grandparents

used to do back in the day was we'd probably go roughly

to the middle of the phone book here,

and just intuitively, if this is an alphabetized phone book

in English, what section am I probably going to

find myself in roughly?

- K?

- Okay. So, I'm in the K section.

Is John going to be to the left or to the right?

- To the left.

- Yeah.

So, John is going to be to the left or the right

and what we can do here, though your phone

does something smarter, is tear the problem in half,

throw half of the problem away,

being left with just 500 pages now.

But what might I next do?

I could sort of naively just start at the beginning again,

but we've learned to do better.

I can go roughly to the middle here.

- And you can do it again. - Yeah, exactly.

So, now maybe I'm in the E section,

which is a little to the left.

So, John is clearly going to be to the right,

so I can again tear the problem poorly in half,

throw this half of the problem away,

and I claim now that if we started with a thousand pages,

now we've gone to 500, 250,

now we're really moving quickly.

- Yeah.

- [David] And so, eventually I'm hopefully dramatically

left with just one single page

at which point John is either on that page

or not on that page, and I can call him.

Roughly how many steps might this third algorithm take

if I started with a thousand pages

then went to 500, 250, 125,

how many times can you divide 1,000 in half? Maybe?

- 10.

- That's roughly roughly 10.

Because in the first algorithm,

looking again for someone like Zoe in the worst case

might have to go all the way through a thousand pages.

But the second algorithm you said was 500,

maybe 501, essentially the same thing.

So, twice as fast.

But this third and final algorithm is sort of fundamentally

faster because you're sort of dividing and conquering it

in half, in half, in half,

not just taking one or two bites out of it out of a time.

So, this of course is not how we used to use phone books

back in the day since otherwise they'd be single use only.

But it is how your phone is actually searching for Zoe,

for John, for anyone else, but it's doing it in software.

- Oh, that's cool.

- So, here we've happened to focus on searching algorithms,

looking for John in the phone book.

But the technique we just used

can indeed be called divide and conquer,

where you take a big problem and you divide and conquer it,

that is you try to chop it up into smaller,

smaller, smaller pieces.

A more sophisticated type of algorithm,

at least depending on how you implement it,

something known as a recursive algorithm.

Recursive algorithm is essentially an algorithm

that uses itself to solve the exact same problem

again and again, but chops it smaller, and smaller,

and smaller ultimately.

[upbeat music]

- Hi, my name's Patricia.

- Patricia, nice to meet you.

Where are you a student at?

- I am starting my senior year now at NYU.

- Oh nice. And what have you been studying

the past few years?

- I studied computer science and data science.

- If you were chatting with a non-CS,

non-data science friend of yours,

how would you explain to them what an algorithm is?

- Some kind of systematic way of solving a problem,

or like a set of steps to kind of solve

a certain problem you have.

- So, you probably recall learning topics

like binary search versus linear search, and the like.

- Yeah.

- So, I've come here complete with a

actual chalkboard with some magnetic numbers on it here.

How would you tell a friend to sort these?

- I think one of the first things we learned was

something called bubble sort.

It was kind of like focusing on smaller bubbles

I guess I would say of the problem,

like looking at smaller segments rather than

the whole thing at once.

- What is I think very true about what you're hinting at

is that bubble sort really focuses on local, small problems

rather than taking a step back trying to fix

the whole thing, let's just fix the obvious problems

in front of us. So, for instance, when we're trying to get

from smallest to largest,

and the first two things we see are eight followed by one,

this looks like a problem 'cause it's out of order.

So, what would be the simplest fix,

the least amount of work we can do

to at least fix one problem?

- Just switch those two numbers

'cause one is obviously smaller than eight.

- Perfect. So, we just swap those two then.

- You would switch those again.

- Yeah, so that further improves the situation

and you can kind of see it,

that the one and the two are now in place.

How about eight and six?

- [Patricia] Switch it again.

- Switch those again. Eight and three?

- Switch it again.

[fast forwarding]

- And conversely now the one and the two are closer to,

and coincidentally are exactly where we want them to be.

So, are we done?

- No.

- Okay, so obviously not, but what could we do now

to further improve the situation?

- Go through it again but you don't need

to check the last one anymore because we know

that number is bubbled up to the top.

- Yeah, because eight has indeed bubbled all the way

to the top. So, one and two?

- [Patricia] Yeah, keep it as is.

- Okay, two and six?

- [Patricia] Keep it as is.

- Okay, six and three?

- Then you switch it.

- Okay, we'll switch or swap those.

Six and four?

- [Patricia] Swap it again.

- Okay, so four, six and seven?

- [Patricia] Keep it.

- Okay. Seven and five?

- [Patricia] Swap it.

- Okay. And then I think per your point,

we're pretty darn close.

Let's go through once more.

One and two? - [Patricia] Keep it.

- Two three? - [Patricia] Keep it.

- Three four? - [Patricia] Keep it.

- Four six? - [Patricia] Keep it.

- Six five?

- [Patricia] And then switch it.

- All right, we'll switch this. And now to your point,

we don't need to bother with the ones

that already bubbled their way up.

Now we are a hundred percent sure it's sorted.

- Yeah.

- And certainly the search engines of the world,

Google and Bing and so forth,

they probably don't keep webpages in sorted order

'cause that would be a crazy long list

when you're just trying to search the data.

But there's probably some algorithm underlying what they do

and they probably similarly, just like we,

do a bit of work upfront to get things organized

even if it's not strictly sorted in the same way

so that people like you and me and others

can find that same information.

So, how about social media?

Can you envision where the algorithms are in that world?

- Maybe for example like TikTok, like the For You page,

'cause those are like recommendations, right?

It's sort of like Netflix recommendations

except more constant because it's just every video

you scroll, it's like that's a new recommendation basically.

And it's based on what you've liked previously,

what you've saved previously, what you search up.

So, I would assume there's some kind of algorithm there

kind of figuring out like what to put on your For You page.

- Absolutely. Just trying to keep you presumably

more engaged.

So, the better the algorithm is,

the better your engagement is,

maybe the more money the company then makes on the platform

and so forth.

So, it all sort of feeds together.

But what you're describing really is more

artificially intelligent, if I may,

because presumably there's not someone at TikTok

or any of these social media companies saying,

"If Patricia likes this post, then show her this post.

If she likes this post, then show her this other post."

Because the code would sort of grow infinitely long

and there's just way too much content for a programmer

to be having those kinds of conditionals,

those decisions being made behind the scenes.

So, it's probably a little more artificially intelligent.

And in that sense you have topics like neural networks,

and machine learning which really describe

taking as input things like what you watch,

what you click on, what your friends watch,

what they click on, and sort of trying to infer

from that instead, what should we show Patricia

or her friends next?

- Oh, okay. Yeah. Yeah.

That makes like the distinction more...

Makes more sense now.

- Nice. - Yeah.

[upbeat music]

- I am currently a fourth year PhD student at NYU.

I do robot learning, so that's half and half

robotics and machine learning.

- Sounds like you've dabbled with quite a few algorithms.

So, how does one actually research algorithms

or invent algorithms?

- The most important way is just trying to think about

inefficiencies, and also think about connecting threads.

The way I think about it is that algorithm for me

is not just about the way of doing something,

but it's about doing something efficiently.

Learning algorithms are practically everywhere now.

Google, I would say for example,

is learning every day about like,

"Oh what articles, what links might be better than others?"

And re-ranking them.

There are recommender systems all around us, right?

Like content feeds and social media,

or you know, like YouTube or Netflix.

What we see is in a large part determined by this kind of

learning algorithms.

- Nowadays there's a lot of concerns

around some applications of machine learning

like deep fakes where it can kind of learn how I talk

and learn how you talk and even how we look,

and generate videos of us.

We're doing this for real, but you could imagine

a computer synthesizing this conversation eventually.

- Right.

- But how does it even know what I sound like

and what I look like, and how to replicate that?

- All of this learning algorithms that we talk about, right?

A lot, like what goes in there is just

lots and lots of data.

So, data goes in, something else comes out.

What comes out is whatever objective function

that you optimize for.

- Where is the line between algorithms that

play games with and without AI?

- I think when I started off my undergrad,

the current AI machine learning

was not very much synonymous.

- Okay.

- And even in my undergraduate, in the AI class,

they learned a lot of classical algorithms for game plays.

Like for example, the A star search, right?

That's a very simple example of how you can play a game

without having anything learned.

This is very much, oh you are at a game state,

you just search down, see what are the possibilities

and then you pick the best possibility that it can see,

versus what you think about when you think about,

ah yes, gameplay like the alpha zero for example,

or alpha star, or there are a lot of, you know,

like fancy new machine learning agents that are

even learning very difficult games like Go.

And those are learned agents, as in they are getting better

as they play more and more games.

And as they get more games, they kind of

refine their strategy based on the data that I've seen.

And once again, this high level abstraction

is still the same.

You see a lot of data and you'll learn from that.

But the question is what is objective function

that you're optimizing for?

Is it winning this game?

Is it forcing a tie or is it, you know,

opening a door in a kitchen?

- So, if the world is very much focused on supervised,

unsupervised reinforcement learning now,

what comes next five, ten years, where is the world going?

- I think that this is just going to be more and more,

I don't want to use the word encroachment,

but that's what it feels like of algorithms

into our everyday life.

Like even when I was taking the train here, right?

The trains are being routed with algorithms,

but this has existed for you know, like 50 years probably.

But as I was coming here, as I was checking my phone,

those are different algorithms,

and you know, they're kind of getting all around us,

getting there with us all the time.

They're making our life better most places, most cases.

And I think that's just going to be a continuation

of all of those.

- And it feels like they're even in places

you wouldn't expect, and there's just so much data

about you and me and everyone else online

and this data is being mined and analyzed,

and influencing things we see and hear it would seem.

So, there is sort of a counterpoint which might be good

for the marketers, but not necessarily good for you and me

as individuals.

- We are human beings, but for someone

we might be just a pair of eyes who are

carrying a wallet, and are there to buy things.

But there is so much more potential for these algorithms

to just make our life better without

changing much about our life.

[upbeat music]

- I'm Chris Wiggins. I'm an associate professor

of Applied Mathematics at Columbia.

I'm also the chief data scientist of the New York Times.

The data science team at the New York Times

develops and deploys machine learning

for newsroom and business problems.

But I would say the things that we do mostly, you don't see,

but it might be things like personalization algorithms,

or recommending different content.

- And do data scientists, which is rather distinct

from the phrase computer scientists.

Do data scientists still think in terms of algorithms

as driving a lot of it?

- Oh absolutely, yeah.

In fact, so in data science and academia,

often the role of the algorithm is

the optimization algorithm that helps you find the best

model or the best description of a data set.

And data science and industry, the goal,

often it's centered around an algorithm

which becomes a data product.

So, a data scientist in industry might be

developing and deploying the algorithm,

which means not only understanding the algorithm

and its statistical performance,

but also all of the software engineering

around systems integration, making sure that that algorithm

receives input that's reliable and has output that's useful,

as well as I would say the organizational integration,

which is how does a community of people

like the set of people working at the New York Times

integrate that algorithm into their process?

- Interesting. And I feel like AI based startups

are all the rage and certainly within academia.

Are there connections between AI

and the world of data science?

- Oh, absolutely.

- The algorithms that they're in,

can you connect those dots for...

- You're right that AI as a field has really exploded.

I would say particularly many people experienced a ChatBot

that was really, really good.

Today, when people say AI,

they're often thinking about large language models,

or they're thinking about generative AI,

or they might be thinking about a ChatBot.

One thing to keep in mind is a ChatBot is a special case

of generative AI, which is a special case of using

large language models, which is a special case of using

machine learning generally,

which is what most people mean by AI.

You may have moments that are what John McCarthy called,

"Look Ma, no hands," results,

where you do some fantastic trick and you're not quite sure

how it worked.

I think it's still very much early days.

Large language models is still in the point of

what might be called alchemy and that people are building

large language models without a real clear,

a priori sense of what the right design is

for a right problem.

Many people are trying different things out,

often in large companies where they can afford

to have many people trying things out,

seeing what works, publishing that,

instantiating it as a product.

- And that itself is part of the scientific process

I would think too.

- Yeah, very much. Well, science and engineering,

because often you're building a thing

and the thing does something amazing.

To a large extent we are still looking for

basic theoretical results around why

deep neural networks generally work.

Why are they able to learn so well?

They're huge, billions of parameter models

and it's difficult for us to interpret

how they're able to do what they do.

- And is this a good thing, do you think?

Or an inevitable thing that we, the programmers,

we, the computer scientists, the data scientists

who are inventing these things,

can't actually explain how they work?

Because I feel like friends of mine in industry,

even when it's something simple and relatively familiar

like auto complete, they can't actually tell me

why that name is appearing at the top of the list.

Whereas years ago when these algorithms were more

deterministic and more procedural,

you could even point to the line that made that name

bubble up to the top. - [Chris] Absolutely.

- So, is this a good thing, a bad thing,

that we're sort of losing control perhaps in some sense

of the algorithm?

- It has risks.

I don't know that I would say that it's good or bad,

but I would say there's lots of scientific precedent.

There are times when an algorithm works really well

and we have finite understanding of why it works

or a model works really well

and sometimes we have very little understanding

of why it works the way it does.

- In classes I teach, certainly spend a lot of time on

fundamentals, algorithms that have been taught in classes

for decades now, whether it's binary search,

linear search, bubble sorts, selection sort or the like,

but if we're already at the point where I can pull up

chat GPT, copy paste a whole bunch of numbers or words

and say, "Sort these for me,"

does it really matter how Chat GPT is sorting it?

Does it really matter to me as the user

how the software is sorting it?

Do these fundamentals become more dated and less important

do you think?

- Now you're talking about the ways in which code

and computation is a special case of technology, right?

So, for driving a car, you may not necessarily need

to know much about organic chemistry,

even though the organic chemistry is how the car works.

So, you can drive the car and use it in different ways

without understanding much about the fundamentals.

So, similarly with computation, we're at a point

where the computation is so high level, right?

You can import psychic learn and you can go from zero

to machine learning in 30 seconds.

It's depending on what level you want to understand

the technology, where in the stack, so to speak,

it's possible to understand it and make wonderful things

and advance the world without understanding it

at the particular level of somebody who actually might have

originally designed the actual optimization algorithm.

I should say though, for many of the optimization

algorithms, there are cases where an algorithm

works really well and we publish a paper,

and there's a proof in the paper,

and then years later people realize

actually that proof was wrong and we're really

still not sure why that optimization works,

but it works really well or it inspires people

to make new optimization algorithms.

So, I do think that the goal of understanding algorithms

is loosely coupled to our progress

and advancing grade algorithms, but they don't always

necessarily have to require each other.

- And for those students especially,

or even adults who are thinking of now steering into

computer science, into programming,

who were really jazzed about heading in that direction

up until, for instance, November of 2022,

when all of a sudden for many people

it looked like the world was now changing

and now maybe this isn't such a promising path,

this isn't such a lucrative path anymore.

Are LLMs, are tools like Chat GPT reason not to perhaps

steer into the field?

- Large language models are a particular architecture

for predicting, let's say the next word,

or a set of tokens more generally.

The algorithm comes in when you think about

how is that LLM to be trained or also how to be fine tuned.

So, the P of GPT is a pre-trained algorithm.

The idea is that you train a large language model

on some corpus of text, could be encyclopedias,

or textbooks, or what have you.

And then you might want to fine tune that model

around some particular task or

some particular subset of texts.

So, both of those are examples of training algorithms.

So, I would say people's perception

of artificial intelligence has really changed a lot

in the last six months, particularly around November of 2022

when people experienced a really good ChatBot.

The technology though had been around already before.

Academics had already been working with Chat GPT three

before that and GPT two and GPT one.

And for many people it sort of opened up this conversation

about what is artificial intelligence

and what could we do with this?

And what are the possible good and bad, right?

Like any other piece of technology.

Kranzburg's first law of technology,

technology is neither good, nor bad, nor is it neutral.

Every time we have some new technology,

we should think about it's capabilities

and the good, and the possible bad.

- [David] As with any area of study,

algorithms offer a spectrum from the most basic

to the most advanced.

And even if right now, the most advanced of those algorithms

feels out of reach because you just

don't have that background,

with each lesson you learn, with each algorithm you study,

that end game becomes closer and closer

such that it will, before long, be accessible to you

and you will be at the end of that most advanced spectrum.