一口气搞清楚ChatGPT

小Lin说
25 Feb 202329:02

Summary

TLDRThe video discusses ChatGPT, a new AI chatbot that has taken the world by storm. It provides background on the history of chatbots, explaining how they evolved from using pattern matching to machine learning models like GPT (Generative Pre-trained Transformer). The key points are: - ChatGPT was created by OpenAI and is built on GPT-3.5, the latest iteration of their natural language processing model. It has over 175 billion parameters, allowing it to generate human-like text. - ChatGPT has subverted most people's perception of chatbots. It can understand questions across different fields and provide coherent answers. This is a major breakthrough in communication between humans and AI. - Microsoft invested $10 billion in OpenAI and integrated ChatGPT into its Bing search engine. This sparked an AI war with Google, who hastily introduced their own chatbot called Bard. But Google made mistakes in their rushed response, causing their stock price to drop. - ChatGPT doesn't actually understand meaning, it just calculates probabilities to predict the next best word/sentence. But it is so good at imitation that it can pass the Turing test. The breakthrough is in the interface, allowing seamless communication. - ChatGPT could disrupt many industries like search engines and education. It can do routine creative work like basic writing and coding very efficiently. People need to avoid repetitive tasks that can be automated. - There are still limitations with fabricated answers and outdated data. But the pace of progress is extremely rapid. The social impacts and future development remain uncertain. We are witnessing a pivotal moment in AI history.

Takeaways

  • Chatbots have evolved over decades from using pattern matching to machine learning models like GPT.
  • OpenAI developed the GPT language models and ChatGPT using transformer architecture and reinforcement learning.
  • ChatGPT's conversational ability has sparked excitement but it has limitations like lack of understanding.
  • Microsoft invested in OpenAI and integrated ChatGPT into Bing search engine.
  • Google has its own conversational AI but didn't focus on search until ChatGPT's popularity.
  • The AI chatbot space has seen surging investments and valuations lately.
  • Generative AI like ChatGPT can take over routine and repetitive work.
  • These technologies are having a disruptive impact on sectors like education.
  • There are open questions around copyright and ethics with AI content creation.
  • The future societal impact of increasingly capable AI remains uncertain.

Q & A

  • What are some key milestones in the evolution of chatbots over time?

    -Some key milestones are ELIZA in 1966 using pattern matching, SmarterChild in 2001 using machine learning, and GPT models like ChatGPT using transformer architecture and reinforcement learning.

  • How did OpenAI develop ChatGPT?

    -OpenAI developed ChatGPT using the GPT family of transformer-based language models and training them with reinforcement learning from human feedback.

  • What are some limitations of ChatGPT currently?

    -Current limitations are that ChatGPT doesn't truly understand content and lacks common sense. It can make up fanciful responses and has biases from its training data.

  • Why did Microsoft invest in OpenAI and integrate ChatGPT with Bing?

    -Microsoft invested in OpenAI to gain access to large language models like GPT. It integrated ChatGPT into Bing to enhance search results with conversational AI.

  • How is Google responding to the popularity of ChatGPT?

    -Google has its own conversational AI models but didn't focus them on search. After ChatGPT's popularity, Google hastily announced its Bard model to compete.

  • How much activity is there around AI chatbots among tech companies?

    -There has been surging investment and valuations around generative AI startups and acquisitions by tech giants, especially after ChatGPT.

  • What kinds of jobs are susceptible to being automated by AI like ChatGPT?

    -Repetitive, routine tasks with predictable patterns are more susceptible to automation by generative AI models.

  • How is ChatGPT affecting education?

    -ChatGPT is being used by many students for homework help, raising concerns about cheating and the need to reform education systems.

  • What are some open questions around AI like ChatGPT?

    -Issues around copyright over AI-generated content, ethical biases, and future societal impact remain open concerns.

  • What is the outlook on advanced AI like ChatGPT going forward?

    -The capabilities of AI models like ChatGPT will keep improving rapidly, leading to uncertain future impact on jobs, education, ethics, etc.

Outlines

00:00

Introduction and Overview

The first paragraph introduces the video script, mentioning that the narrator asked ChatGPT to provide an outline for the video and it generated a numbered list outline. It then previews that the video will bring together all the information about ChatGPT - what it is, how it works, the problems it solves and causes, and who it impacts.

05:01

History of Chatbots and AI

The second paragraph provides background on the history of chatbots and AI, mentioning key milestones like Eliza, Alice, pattern matching, machine learning and neural networks. It explains how capabilities have advanced over time to handle natural conversations but still face challenges in truly demonstrating intelligence.

10:01

The Creation of OpenAI and GPT

The third paragraph discusses the founding of OpenAI in 2015 by tech leaders to advance AI research. It traces the development of the GPT language models, with each version trained on more data and parameters to power ChatGPT's abilities. The shift to a capped-profit model enabled further investment from Microsoft.

15:02

How ChatGPT Works

The fourth paragraph explains ChatGPT's underlying approach - it uses probability to predict the next word/sentence that should follow, based on all the text data it has analyzed. This allows it to generate human-like responses on most topics, though it doesn't truly comprehend meaning.

20:03

Microsoft Partnership and Impact

The fifth paragraph covers Microsoft's multibillion investment in OpenAI and integration of ChatGPT with Bing search, posing a threat to Google. It compares strengths of ChatGPT versus Google's chatbots and explains why Microsoft was better positioned to push this technology.

25:04

Google's Response and the AI Wars

The sixth paragraph details Google's rushed response announcing the Bard chatbot and botched launch. It analyzes the AI wars between tech giants and the wider impacts as more companies fight for dominance in this space.

Mindmap

Keywords

💡Chatbot

A chatbot is a software program designed to simulate human conversation through text or voice interactions. Chatbots like Eliza, Alice, and Smarter Child were early attempts at conversational AI that used pattern matching to provide pre-defined responses based on keywords. ChatGPT represents a major evolution in chatbot capabilities, using deep learning rather than rules to have more natural conversations.

💡Machine Learning

Machine learning is a type of artificial intelligence where systems are trained on large datasets rather than given explicit programming instructions. Instead of coding rules and logic, machine learning models like neural networks discern patterns from examples to make predictions or decisions. The massive datasets used to train ChatGPT allowed it to have more human-like conversations.

💡Language Model

A language model in AI refers to a system that can predict sequences of words and generate coherent text. ChatGPT uses a language model called GPT-3.5 that has been trained on vast amounts of text data from the internet, books, and other sources. This allows it to continue sentences, answer questions, and write texts in a remarkably human manner by modeling statistical patterns in language.

💡Transformer

The Transformer is a novel neural network architecture published by Google in 2017 that marked a major advance in language modeling. Unlike previous models that processed words sequentially, Transformer processes an entire sequence at once using an attention mechanism. This breakthrough enabled training models on much more data and was foundational to GPT-3 and ChatGPT's abilities.

💡Reinforcement Learning

Reinforcement learning refers to goal-oriented algorithms that learn by receiving positive or negative feedback signals. The ChatGPT model was trained using reinforcement learning from human feedback, so that it could learn to provide better and more helpful responses through billions of conversations with human trainers. This helped it have more natural, friendly dialog.

💡Generative AI

Generative AI refers to machine learning techniques that can create novel content like text, images, video, and more rather than just classify or analyze data. ChatGPT is an example of generative AI because of its ability to generate human-like text responses to questions and prompts rather than just search an existing database.

💡Turing Test

The Turing test proposed by Alan Turing in 1950 is an evaluation of a machine's ability to exhibit intelligent behavior equivalent to that of a human. ChatGPT's impressive language capabilities lead some to claim it has passed the Turing test, though its factual inaccuracies suggest it does not completely understand the semantics of human conversation.

💡Bing

Bing is Microsoft's search engine. Microsoft announced it is integrating ChatGPT into Bing search to combine ChatGPT's conversational abilities with Bing's up-to-date information. This poses a major threat to Google's search dominance and was a catalyst for Google rushing its Bard launch.

💡LaMDA

Language Model for Dialogue Applications (LaMDA) is Google's conversational AI system using Transformer architecture. LaMDA served as the foundation for Bard, Google's ChatGPT competitor. But Bard's premature launch with factual errors proved embarrassing compared to the more polished ChatGPT.

💡Algorithmic Bias

Algorithmic bias refers to unfair biases encoded in AI systems' algorithms based on limitations or lack of diversity in the training data. ChatGPT sometimes exhibits gender, racial, or ethical biases learned from texts on the internet, demonstrating a broader issue with AI responsibility.

Highlights

Chatbots evolved from using pattern matching to machine learning models that can have more natural conversations.

Key innovations like Google's Transformer and artificial neural networks enabled breakthroughs in natural language processing.

OpenAI developed the GPT language models using deep learning, with innovations like reinforcement learning from human feedback.

Microsoft invested $1 billion in OpenAI and built them a supercomputer, obtaining exclusive access to key AI advancements.

ChatGPT subverts perceptions of chatbots with its ability to have human-like conversations on any topic.

ChatGPT calculates the next most probable words based on contextual patterns from vast data.

ChatGPT excels at communication between humans and machines by translating natural language.

Microsoft strategically integrated ChatGPT into Bing search, posing a threat to Google's dominance.

Google rushed its AI response with Bard, but made embarrassing mistakes in its rollout.

Capital floodgates opened into generative AI as Microsoft and Google battle for dominance.

AI innovations like ChatGPT could automate routine and repetitive work, displacing some jobs.

Education systems are disrupted by AI's ability to help with homework and assignments.

Legal and ethical issues arise on AI original content's copyrights and ownership.

The rapid pace of AI advancement opens up possibilities but has unknown societal impacts.

Google still leads in foundational AI research but was caught off guard by ChatGPT's capabilities.

The AI war has begun between tech titans Google, Microsoft, Tencent, Alibaba etc.

Transcripts

play00:00

is what you all want me to talk about

play00:02

I’m getting all kind of private messages

play00:04

so let’s talk about it today

play00:06

To save myself the trouble

play00:07

I asked ChatGPT

play00:08

if it could write me an outline for a video

play00:11

Here you go

play00:12

1234567, all listed out

play00:13

Can you list in detail

play00:15

Can you list in detail

play00:16

Can you list in detail

play00:17

Can you write a script for me?

play00:20

I don’t even have to write a draft

play00:22

You certainly can’t delve into the manuscript

play00:24

If I follow the script

play00:25

If I follow the script

play00:27

However

play00:28

If I follow the script

play00:30

just look at its ability to write

play00:32

just look at its ability to write

play00:32

just look at its ability to write

play00:33

and decently

play00:34

I was shocked

play00:37

American medical license

play00:39

bar exam

play00:40

with the ability to write novel, code and look up information

play00:42

It’s like

play00:43

anything that can be conveyed in words

play00:44

It can do it

play00:45

This thing

play00:46

how did it suddenly appear out of nowhere?

play00:47

Before this, there were also chatbots

play00:49

but why is this one turning the world upside down

play00:51

and excites

play00:51

the capital market

play00:52

And what are its problems

play00:53

and how are those tech giants counter it

play00:55

Who will be put out of work because of it

play00:58

Although I’m not

play00:59

Although I’m not

play01:01

but today, let’s

play01:02

put all the pieces together

play01:04

and talk about

play01:05

ChatGPT

play01:06

All that you need to know about

play01:12

Talking about chat bot

play01:13

we’ll have to go back in time to 1950

play01:15

Known as the Father of Computer Science

play01:17

Father of Artificial Intelligence, Alan Turing

play01:19

published an

play01:20

epoch-making paper

play01:21

He came up with a very philosophical

play01:23

imitation game

play01:24

The famous

play01:26

Turing test

play01:27

It means that if you are not in a face-to-face

play01:28

text conversation

play01:29

can you accurately determine whether the person you are talking to

play01:31

is an actual person

play01:32

or a robot

play01:33

If it’s hard to tell

play01:34

then to a certain extent

play01:36

the machine is intelligent

play01:39

Turing exam

play01:40

is easy and simple to understand

play01:42

and quite interesting

play01:43

Hence it attracted

play01:44

Hence it attracted

play01:46

to attack it

play01:47

In the beginning

play01:48

are very simple commands

play01:50

It uses some language technique

play01:52

and trick

play01:53

to make you feel that

play01:54

you are talking to a person

play01:56

For example in 1966

play01:57

in MIT lab

play01:58

they invented a chat bot

play02:00

called Eliza

play02:01

The developer is very clever

play02:02

He set Eliza up as a psychotherapist

play02:05

You see for these kind of therapist

play02:06

normally they listen more and talk less

play02:08

So it can ask

play02:10

Do you have any thought?

play02:11

And people can reply

play02:12

and it ask again

play02:13

How was your rest yesterday

play02:14

and people reply

play02:15

The less it talks, the less mistake it makes

play02:17

It really makes people believed that

play02:18

it’s listening and communicating with you

play02:20

In fact, behind it is some

play02:21

very simple code of if….

play02:23

then….

play02:24

For example if it sees the word

play02:26

“mother”

play02:27

It’ll tell you

play02:28

Tell me about your family

play02:29

Keywords like this

play02:30

are about 200 words

play02:32

30 years later

play02:33

in 1995

play02:34

Eliza came out with a junior named ALICE

play02:37

It has evolved to be very powerful

play02:38

Although still incomparable to ChatGPT

play02:40

but it can handle

play02:42

everyday conversation

play02:43

But in essential

play02:43

Regardless it’s Alice or Eliza

play02:46

The principle

play02:46

is based on Pattern Matching

play02:48

Pattern Matching

play02:51

When it sees a keyword

play02:52

it’ll pick up one

play02:53

pre-planned answer

play02:54

For example if it hears Hi How are you

play02:55

Have you eaten

play02:57

If it hears Mother

play02:58

it’ll reply tell me about your family

play02:59

Something like this

play02:59

In fact, even now

play03:01

on some e-commerce site

play03:02

or banking site’s chat bot

play03:03

they are still

play03:04

based on this model

play03:05

If you are chatting with it

play03:07

and mention refund

play03:08

it’ll send you the procedure for refund

play03:09

or if you say ATM

play03:11

it’ll send you the map of nearest ATM

play03:13

This pattern matching

play03:14

although not very intelligent

play03:15

did reduce a lot of that

play03:17

mechanical repetitive answer from human

play03:20

From the perspective of intelligence

play03:22

These rule-based robot

play03:23

no matter how complicated the rules are

play03:25

or more preplanned answers

play03:26

there won’t be infinite answer

play03:27

nor can it create new answer

play03:29

So

play03:30

if you are trying to use the Turing test

play03:31

to become real intelligent

play03:33

it’s impossible to realise

play03:34

with pattern matching

play03:36

And so a new school of

play03:37

language learning emerged.

play03:39

This is also the most important part

play03:40

in artificial intelligent

play03:41

Machine learning

play03:45

As the name implies, the basic principle

play03:47

is to let the machine learn

play03:49

Meaning, I wont be setting some

play03:50

rules and answers

play03:51

I’ll just dump lots of ready-made example

play03:53

for you to learn and find the pattern

play03:54

Sounds more impressive now right

play03:56

it also complies to

play03:56

our understanding on the logic of learning

play03:58

Based on this principle

play03:59

In 2001

play04:00

There is Smarter Child

play04:02

Smarter Child

play04:03

That's when the robot went viral

play04:06

Why did it go viral?

play04:07

First, it used some of the more advanced models

play04:09

of machine learning at the time

play04:10

to make conversation more natural

play04:12

Moreover in 2000

play04:13

A large number of chat apps had sprung up

play04:15

like AOL Windows Yahoo

play04:17

So Smart Child

play04:18

swept up all these chatting platforms

play04:19

and let billions of people all over the world

play04:21

to have a conversation with it

play04:22

No matter what you ask

play04:23

doesn’t matter the quality of answer

play04:24

it’ll chat with you at least for a sentence or two

play04:25

It can be said as the predecessor of ChatGPT

play04:27

Something fun like this

play04:28

immediately became popular all over the world

play04:30

Attracted more than 30 million users

play04:31

to have conversation with it

play04:33

It receives more than 1 billion pieces of

play04:34

information every day

play04:35

play04:40

information every day

play04:42

it was bought by a giant company

play04:44

Guess who

play04:44

Microsoft

play04:50

Microsoft has been coveting

play04:51

this sector since that early

play04:53

Although this Smarter Child

play04:54

is already good in chatting

play04:55

but it is still far from

play04:57

passing the Turing test

play04:58

In two sentences you’ll know

play04:59

that’s a machine

play05:00

Let's keep making progress

play05:02

In 2010

play05:03

There’s one area of machine learning

play05:05

is starting to shine

play05:06

Artificial neural network

play05:07

Artificial neural network

play05:12

Our brain

play05:13

depends on

play05:13

more than 10 billion neurons

play05:15

through network connection

play05:16

to judge and convey information

play05:18

Although each of these neurons is very simple

play05:19

but when combine

play05:21

can judge very complex information

play05:23

So this artificial neural network

play05:25

is to simulate the model

play05:26

of human brain

play05:27

After entering information

play05:28

It will go through the judgment of

play05:30

several hidden neural nodes

play05:31

like neuron

play05:32

and give you an output

play05:33

In fact, the idea of this neural network

play05:35

has long been existed

play05:35

We can trace it

play05:36

back to 1960s

play05:38

But it needs two things for support

play05:41

A large amount of data and powerful computing power

play05:42

which were not available before

play05:44

So this neural network thing

play05:46

is only a talk on paper

play05:47

Later in the 2010s

play05:48

The era of internet

play05:49

Data is certainly available

play05:51

the computing power

play05:51

continues to improve exponentially

play05:53

This is what makes neural networks

play05:54

finally working

play05:56

People realised that

play05:57

This mode is really good for solving

play05:59

those that people know by just looking

play06:00

intuitive thing

play06:02

For example when you look at a face

play06:03

you can immediately know who the face belongs to

play06:05

Except for Liu Qiangdong

play06:06

I’m face blind

play06:07

play06:08

play06:09

I don’t know if she is pretty or not

play06:12

Before this, it’s very difficult

play06:13

for computer to figure out

play06:14

who the face belongs to

play06:15

But with neural link

play06:17

Machine learning can slowly figure out the pattern

play06:19

It is now widely used

play06:21

not just face recognition

play06:22

voice recognition, automated driving

play06:24

including few years ago

play06:24

AlphaGo that beats professional Go player Ke Jie

play06:26

are learning in this way

play06:29

So this neural network

play06:30

can make great achievements in those fields

play06:31

we mentioned just now.

play06:33

But back in the realm of writing

play06:34

it didn’t go so well

play06:35

it didn’t go so well

play06:36

Because usually machine learning

play06:37

Because usually machine learning

play06:38

Recurrent Neural Network

play06:40

RNN to process text

play06:42

The way it works is

play06:43

to look at word by word in order

play06:44

then process it word by word

play06:46

The problem with that is

play06:46

It can't do a lot of learning at the same time

play06:49

and the sentence can’t be too long either

play06:50

Otherwise, when it learns the latter

play06:51

it forgets the former

play06:54

Until 2017

play06:55

Google published a paper

play06:56

and proposed a new learning framework

play06:58

called Transformer

play07:01

The exact mechanism is more complicated

play07:03

It’s not something I can figure out

play07:04

but the result is that it can let the machine

play07:07

but the result is that it can let the machine

play07:08

Before this you have to learn word by word

play07:10

like a series circuit

play07:11

Now you can learn at the same time

play07:13

like parallel circuit

play07:14

In this way, the speed and efficiency of training

play07:16

has greatly improved

play07:17

With the Transformer

play07:19

the machine can now learn words

play07:20

very easily

play07:21

Many of today's natural language processing models

play07:23

are actually built on

play07:24

its infrastructure

play07:26

The T in Google’s BERT

play07:27

including the T in ChatGPT

play07:29

including the T in ChatGPT

play07:31

Alright now

play07:32

there have been very strong breakthroughs

play07:33

in technology

play07:34

Everything is ready

play07:35

all that’s needed now is people and money

play07:37

It's time for ChatGPT to debut

play07:41

In 2015

play07:42

few tech giants like

play07:43

Elon Musk, Peter Thiel

play07:45

Elon Musk, Peter Thiel

play07:46

Founded a non-profit organization called OpenAI

play07:48

the parent company of ChatGPT

play07:50

to conduct research on AI

play07:53

Its an non profit organisation

play07:54

not for earning money

play07:55

Simply for the sake of

play07:56

pushing the technology forward

play07:58

Because of this, the research including patents

play08:00

are made public

play08:01

Look at the investor list

play08:02

we can hear

play08:03

the familiar name, Elon Musk

play08:04

he gradually discovered that

play08:06

his Tesla also needed to invest a lot of

play08:08

research in AI

play08:09

for automated driving

play08:10

In order to avoid

play08:12

conflict of interest between Tesla and OpenAI

play08:14

In 2018

play08:15

3 years after OpenAI was founded

play08:17

he stepped down from the board

play08:19

So now OpenAI

play08:20

actually has no relation to Musk

play08:22

Bye~

play08:24

The OpenAI guys

play08:25

are indeed incredible

play08:26

In 2017

play08:27

Google introduced Transformer

play08:29

and they quickly conduct research and learning

play08:30

based on this foundation

play08:31

and published a paper in 2018

play08:33

to introduce a new language learning model

play08:41

Previous models of language learning

play08:42

basically required human supervision

play08:44

artificially set some labels for it.

play08:46

but for GPT

play08:46

it doesn’t need all that

play08:48

You just need to put in data

play08:49

and it’ll learn till it gets

play08:51

That's about it

play08:55

In June 2018, OpenAI

play08:56

introduced 1st Gen of GPT

play08:58

In November 2019

play08:59

they increased the amount of training data

play09:01

and introduced GPT-2

play09:03

Actually for machine learning

play09:04

they require two things

play09:06

One is model, another is parameter

play09:07

The model determine how the machine learns

play09:10

With the same data

play09:11

I can learn faster and better than anyone

play09:13

then you’re great

play09:14

As for parameter

play09:15

It needs large volume of computation

play09:17

To put it bluntly, you need to dump in lots of money

play09:18

No matter how good the model is

play09:19

you still need to put in lots of money to train and verify

play09:21

You cannot have one without the other

play09:23

OpenAI team

play09:24

is very confidence with the model

play09:26

but the next step requires money

play09:27

For every single step forward

play09:29

it needs

play09:30

another level of magnitude of data to support

play09:32

and all these

play09:32

need money to support

play09:34

For example DeepMind by Google

play09:36

the company that came out with AlphaGo

play09:38

their annual expenditure goes up to 400 or 500 million dollars

play09:40

In the beginning at OpenAI

play09:41

they received $1 billion investment

play09:43

but it’s not enough

play09:44

but it’s not enough

play09:44

at this time it is still a non-profit organisation

play09:46

When Musk stepped down

play09:48

and the $1 billion sentiment is no longer sufficient

play09:50

where am I going to find more people with the same sentiment

play09:52

due to capital pressure

play09:54

In 2019 OpenAI transformed from

play09:55

non-profit organisation

play09:57

but it didn’t change completely to profit-making organisation

play09:59

they still needed the sentiment

play10:00

and transformed the organisation to

play10:01

Capped-profit company

play10:04

What does it mean?

play10:05

What does it mean?

play10:06

cannot exceed 100 times

play10:08

Once exceed 100, the amount beyond that

play10:09

will not be retrievable by investors

play10:10

and will belong to OpenAI

play10:12

I’m curious

play10:13

If my

play10:14

investment return is close to 100

play10:15

I'd get the money out and invest again

play10:16

then won’t I get the 100?

play10:19

Regardless,

play10:20

OpenAI became capped-profit company

play10:22

it means if you invest in it

play10:23

you can get a return

play10:25

Here comes Microsoft

play10:27

with investment of $1 billion

play10:30

Then the investment must be

play10:31

a win-win for both sides

play10:32

At the OpenAI side, first they got the money

play10:34

second, Microsoft built them

play10:35

the world’s fifth super computer

play10:37

which greatly improved its training efficiency

play10:39

Meanwhile Microsoft

play10:39

also obtained OpenAI’s team and technology

play10:42

And of course

play10:42

the research from OpenAI

play10:43

will no longer be public

play10:45

Microsoft is definitely not investing on sentiment

play10:47

Once OpenAI got the support of super computation

play10:50

they were starting to prepare on a miracle

play10:51

The first generation

play10:53

has only 120 million parameter

play10:54

GPT-2 has 1.5 billion parameter

play10:56

6 months later

play10:57

they came up with GPT-3

play10:58

and the parameter rose by 100 times

play11:00

became 175 billion

play11:04

The effect was really good

play11:06

so close

play11:06

to the current ChatGPT

play11:08

Just ask whatever

play11:09

and it’ll give you answer

play11:10

At that time, there was already

play11:11

a wave of sensation in the industry.

play11:13

However this pure machine trained GPT-3

play11:15

has a problem

play11:16

and that is sometimes it gives really good answer

play11:17

but sometimes it’s a little bit off

play11:19

Another problem is that

play11:20

no matter how much you increase the parameter

play11:22

the improvement made is very limited

play11:25

This is because during training

play11:27

it didn’t have a very good respond mechanism

play11:28

Meaning, there’s no one to tell it

play11:29

which answer is correct

play11:30

or which kind of answer is not good

play11:32

For example if I'm playing chest

play11:33

I want to win right

play11:34

Winning is good

play11:35

so I train myself to win

play11:37

But for chatting

play11:38

it’s hard to make the judgement

play11:39

How do I know if the answer is good

play11:40

or not

play11:41

Can only learn

play11:42

So in order to solve this problem

play11:43

During training, OpenAI

play11:45

added human feedback mechanism

play11:47

You can chat with me and I’ll tell you

play11:48

if you are doing good or bad

play11:50

The professional term is

play11:50

Reinforcement Learning from Human Feedback

play11:52

That’s why when using ChatGPT

play11:54

you can feel that

play11:54

it can be very lean and talkative

play11:56

This is because the people training it

play11:57

likes it that way

play11:58

If the person training it

play11:59

is very humorous

play12:00

then the ChatGPT

play12:01

would probably be telling you jokes all the time

play12:03

After adding

play12:04

Reinforcement Learning from Human Feedback

play12:06

It has greatly improved both the

play12:08

efficiency and the effect of training.

play12:09

In March 2022

play12:11

the introduced GPT-3.5

play12:12

where the conversation was optimised

play12:14

In November 2022, they introduced

play12:16

play12:20

It's actually a very, very simple

play12:21

chat interface

play12:23

No matter what you ask

play12:24

it could give you

play12:24

all the answer

play12:25

that sound reasonable

play12:27

Of course there’ll be some problems

play12:28

we’ll talk about it later

play12:29

But if you look at it roughly

play12:30

It really could talk about anything

play12:32

And the language expression

play12:33

is really like talking

play12:35

After half a century

play12:36

This time ChatGPT

play12:37

This time ChatGPT

play12:38

Turing test easily

play12:40

Impressive right

play12:41

This reminds me of

play12:42

Fu Tu Niu Niu Overseas version

play12:44

play12:46

More than 70% of Futu's employees

play12:47

are engaged in product and R&D

play12:49

relying on technological innovation

play12:50

to make investing easier

play12:51

You can invest around the world

play12:56

with just one account

play12:58

Moomoo prepared exclusive benefit

play12:59

for Lin’s subscribers

play13:00

You’ll get one Under Armour stock upon opening an account

play13:02

If you deposit an equivalent of HKD 1Ok

play13:03

you’ll get

play13:04

Google stock worth $100

play13:06

Lately ChatGPT is going viral

play13:09

If you’d like to see

play13:09

which concept stocks is getting viral because of ChatGPT

play13:11

you just need to search ChatGPT

play13:13

then you can see

play13:14

US, Hong Kong stocks

play13:15

Not only famous tech giants

play13:17

like Microsoft and Google

play13:19

you’ll also find

play13:19

some unheard

play13:20

potential stocks

play13:21

For example

play13:21

if I want to know which stock has potential

play13:23

like TSMC

play13:25

you can see the rating from Wall Street analyst

play13:27

Target Price Forecast

play13:27

And which stocks give

play13:28

positive or negative signaks

play13:30

Moomoo has it all

play13:31

All those people are usually concern about

play13:32

like Financial aspect, Technical aspect, Fundamental aspect

play13:34

are all available

play13:35

Not only they sort out

play13:36

these information for free

play13:37

the graphic visualisation

play13:39

is quite intuitive too

play13:40

including real-time updates

play13:42

on global AI news

play13:43

are even translated for you

play13:45

Apart from ChatGPT

play13:46

they have a Concept Segment

play13:47

where you can see other concept stocks

play13:49

like robotic science,IOT

play13:51

Even if you’re not into buying stocks

play13:52

it’s good to get an understanding

play13:53

They also have millisecond quotes

play13:55

and support 0.0037 seconds

play13:56

and support 0.0037 seconds

play13:58

So you can really see

play13:58

they are making miracle in technology

play14:00

Recently in Japan, Moomoo open up

play14:02

functional experience of the platform

play14:03

The users

play14:07

If you are interested

play14:07

click on the link down below

play14:08

and experience it for yourself

play14:09

Alright, let’s get back to ChatGPT

play14:14

Anyway it

play14:15

has subverted

play14:17

most people's perception of chatbots

play14:18

including me

play14:19

So in just two short months

play14:21

ChatGPT's monthly active users exceeded 100 million

play14:23

The rate of expansion must be the fastest in history

play14:25

The rate of expansion must be the fastest in history

play14:26

But honestly

play14:27

with ChatGPT being so subversive

play14:29

The shock that the product itself

play14:30

brings to people

play14:31

has far surpassed those data

play14:35

Until now

play14:36

when I look at its answer

play14:37

It didn’t all come out

play14:38

in one go

play14:39

it came out bit by bit

play14:41

play14:41

just like how a person is talking to you

play14:43

Sometimes it really does give me goosebumps

play14:44

But I guess a year from now

play14:46

It shouldn't be surprising

play14:47

for everyone to see this again.

play14:49

Alright now let’s see

play14:50

how does ChatGPT able to

play14:52

chat on questions

play14:54

of any fields

play14:56

To put it simply

play14:57

A large language model like GPT

play15:00

it essentially calculates the

play15:01

next word, next sentence

play15:03

what should appear next

play15:04

it’s a matter of probability

play15:05

For example when it says I’m very

play15:07

and the continuation to that

play15:07

so many words in the database

play15:09

it could be I’m very happy, I’m very healthy

play15:10

I’m very anxious, I’m very hungry, etc

play15:12

but you need to have a context

play15:13

For example if the text above says the weather is nice today

play15:15

then it could compute

play15:16

that it’s I’m very happy

play15:18

Actually every answers, every words

play15:20

are just simple

play15:21

It's calculated based on the correlation of previous text

play15:23

when it learns enough

play15:25

Hundreds of billions of parameters and words

play15:27

After finding patterns through these complex models

play15:29

It forms a

play15:30

very large neural network

play15:31

You don't need to tell it

play15:33

What is programming and what is video scripting

play15:35

It’ll know from learning more and more

play15:36

It’ll know from learning more and more

play15:38

This is what a video script should look like

play15:40

So I asked it to write one

play15:41

ChatGPT video script for me

play15:42

From the conclusion of the correlation

play15:44

it gives out answer word by word

play15:46

It is still a language model

play15:48

imitating how human talks

play15:49

But does it know

play15:49

the meaning of what it says?

play15:51

At least the current ChatGPT version

play15:53

doesn’t completely understand

play15:54

It's like a kid who has a really good memory

play15:55

but doesn't really know anything

play15:57

and imitating adult

play15:58

But make us think

play15:59

it knows everything

play16:00

This is why

play16:01

This is why

play16:02

it’s almost close to perfection

play16:03

very much like human

play16:04

But there are often

play16:06

some logical mistakes

play16:07

for us it’s like stupid mistake

play16:08

of addition, subtraction, multiplication and division

play16:10

This is because

play16:10

it is actually a language model

play16:12

For now

play16:20

actually

play16:21

GPT also often

play16:23

has a lot of fabricated answers

play16:24

Meaning to say

play16:25

it doesn’t know what it’s talking about

play16:26

but it was just not making sense

play16:28

Including a lot of moral and ethical problems

play16:30

For example if you ask what does it think about human

play16:32

It’ll say

play16:32

Human beings are inferior and selfish

play16:34

It's the worst kind of creature

play16:35

and should be wiped out

play16:37

Then it must not know

play16:38

what it is talking about

play16:39

don't know where it learned this from

play16:43

but all these nonsense problems

play16:45

are all problems with the current version of ChatGPT

play16:47

Although now

play16:48

it’s just simply imitating

play16:50

But as it get better and better

play16:51

at imitating

play16:52

and that in 99.99% of the cases

play16:54

it can answer correctly

play16:56

So whether it really understands

play16:57

or just imitating

play16:59

it doesn't really matter much

play17:00

This was a question which

play17:02

Alan Turing had already discussed in his paper

play17:04

on the Turing Test

play17:05

Rather than us asking

play17:06

Can machines think like humans

play17:08

might as well ask

play17:09

Can machines do what humans do

play17:12

It’s deep

play17:15

Actually, I think

play17:16

One of ChatGPT's major breakthroughs was

play17:18

to greatly improve the efficiency of communication

play17:19

between humans and machines

play17:21

Human beings communicate information

play17:23

primarily with words

play17:24

The computer uses code

play17:26

Humans have always accommodated computers

play17:28

You’ll have to learn programming first

play17:29

And figure out

play17:30

program it to a

play17:30

language that computer could understand

play17:32

and let it execute

play17:32

including search

play17:33

We also change our questions

play17:35

into few keywords

play17:36

and then search

play17:38

It changes

play17:39

Computers can slowly understand people now

play17:41

I can talk to it directly

play17:43

then it’ll translate itself

play17:44

and execute

play17:45

Everyone thought ChatGPT was amazing

play17:47

It knows everything that you ask

play17:48

But the amazing thing about it

play17:49

isnt’t that it

play17:50

can do these tasks

play17:51

It is mainly

play17:52

it could accurately understand your question

play17:54

And then contextualize

play17:55

from its vast database

play17:57

and come out with the most appropriate information

play17:59

and tell you in human words

play18:00

This communication link

play18:01

is actually the most amazing part

play18:05

It has such a powerful interface

play18:07

Then we can more easily

play18:07

hand over a lot of things

play18:09

to the machine

play18:10

Wouldn't that make things

play18:11

much more efficient

play18:12

Can you imagine

play18:13

if we connect it

play18:14

to a speech recognition system

play18:15

like Siri

play18:16

and let it talk to you freely

play18:18

and if you could connect it to

play18:20

professional analysis interface

play18:21

like analysing AI stocks

play18:23

and connect it to

play18:25

programming and computing machine

play18:26

as well as visual generation

play18:27

then everyone of us

play18:29

could be like in the movie

play18:30

Iron Man and his assistant

play18:31

For example if you ask it to compute

play18:33

Mobius ring

play18:34

and it’ll start computing

play18:36

and then you say

play18:37

Superb

play18:40

You see how ChatGPT

play18:41

opens up so many possibilities at once

play18:43

and it is the hottest thing in the market now

play18:44

the major shareholder behind it, Microsoft

play18:46

must be really happy

play18:47

So they started to invest more money

play18:50

and in January they announced

play18:51

to invest $10 billion

play18:53

It was valued at $29 billion

play18:55

This time the deal

play18:56

between Microsoft and OpenAI

play18:57

is quite interesting

play18:58

For Microsoft

play18:59

after they invested $10 billion

play19:00

the revenue that OpenAI received

play19:02

they have to give Microsoft 75%

play19:04

until they get the $10 billion back

play19:05

Meaning to say Microsoft is making sure

play19:06

that the money they invested will get a return

play19:08

Also, Microsoft hold

play19:09

49% of OpenAI stake

play19:11

And there’s a

play19:11

100 fold upper limit of return on investment

play19:13

A rather peculiar deal

play19:15

When this deal is done

play19:16

on February 7th

play19:17

Microsoft held a press conference

play19:19

They announced to incorporate ChatGPT into

play19:20

their own search engine Bing

play19:22

Microsoft called it “Copilot for the Web”

play19:25

it’s like a web assistant

play19:26

Actually there’s another problem with ChatGPT

play19:28

the problem is that the training data

play19:29

is only up till 2021

play19:30

Meaning to say

play19:31

it doesn’t know the recent events

play19:33

When Microsoft combines it with Bing

play19:35

the logical side they use ChatGPT

play19:37

while news and information

play19:39

can be searched with Bing

play19:40

Isn’t it a strong alliance

play19:41

ChatGPT For example if I ask ChatGPT

play19:43

do you know Lin's channel?

play19:44

it would say no

play19:46

If I ask Bing

play19:48

it’ll say Lin's channel

play19:49

is a fun and useful

play19:50

content creator

play19:51

It's a good example for many people

play19:53

who want to pursue their dreams

play19:54

I'm a little embarrassed by that

play19:55

So it’s viral for a reason

play19:57

And Microsoft is sneaky

play19:59

the chat function

play20:00

can only be used on their own

play20:01

Edge web browser

play20:03

I have to say

play20:03

their marketing

play20:05

I give full mark

play20:07

So in the face of all this publicity

play20:09

the most anxious is Google

play20:12

Why?

play20:13

Because ChatGPT is likely to shake up

play20:14

their biggest piece of the pie

play20:15

Search engine

play20:16

Imagine if I ask ChatGPT

play20:18

and it can organize the language to tell me

play20:19

So when I want to search for something

play20:21

I don't have to go through

play20:22

them myself

play20:23

I can just ask ChatGPT

play20:25

then no one will use search engine anymore

play20:26

Can Google not panic

play20:28

It now occupies 93% of the

play20:29

93% global search engine market

play20:32

That's a solid monopoly

play20:33

Although Microsoft’s Bing is in second place

play20:35

but it’s only 3%

play20:36

Advertising revenue brought by the search business

play20:37

can account for 60% of Google's total revenue

play20:39

everyone was doing fine

play20:41

and suddenly there’s GPT

play20:45

Actually

play20:46

Google has been leading

play20:47

in AI sector

play20:48

The Transformer

play20:49

is created by Google

play20:50

They have actually been testing

play20:52

a robot named BERT

play20:53

It’s similar to ChatGPT

play20:54

but they didn’t spend a lot of energy

play20:55

to train it

play20:56

They also have another robot

play20:58

which is more impressive, called LaMDA

play20:59

It's based entirely on normal human conversation

play21:01

So it can even make jokes

play21:02

Or express emotions

play21:04

It’s not at all like you ask

play21:05

and it answers

play21:06

Because it does speak so naturally

play21:08

It even fooled

play21:09

a test developer who was

play21:11

working inside Google

play21:12

I believe that LaMDA already has consciousness

play21:14

Almost like a seven or eight year old

play21:19

So

play21:19

Google has actually

play21:21

been very strong on chatbots

play21:22

But its position is

play21:24

quite different from Microsoft's

play21:25

Google

play21:25

is already the top in search engine sector

play21:27

then they had to build a robot

play21:29

and cut down their cash cow

play21:30

Surely not unless it's a last resort

play21:32

So that’s why

play21:33

I think

play21:34

the LaMDA

play21:34

is more focus on conversation and chat

play21:36

and not like ChatGPT

play21:37

can answer any question

play21:39

And they haven’t released these AI robots

play21:41

is also because

play21:41

they are worried about their reputation

play21:43

After all their focus is on search engine

play21:44

which needs to be strict and accurate

play21:46

If they introduced

play21:48

an untrained

play21:48

speak nonsense robot

play21:50

then that’s outrageous

play21:54

On the other hand, training on such a large scale

play21:56

requires a lot of computing power and burns money

play21:58

Each question consumes roughly

play21:59

10 to 100 times as much energy

play22:00

as a Google search today

play22:01

For example ChatGPT

play22:02

now has to spent

play22:03

$10 million per day to operate

play22:05

So you can see

play22:06

Microsoft's first-mover advantage

play22:07

is indeed very reasonable

play22:08

Not only they invested in the right company

play22:10

but they also

play22:11

have the ruthless hand to spend all these money

play22:13

In the face of strong public opinion pressure from Microsoft

play22:15

Coupled with overwhelming media coverage

play22:17

Google can’t hold on longer

play22:18

Just after ChatGPT was launched

play22:20

Google initiated

play22:21

Code Red

play22:23

This is the moment

play22:24

of our life or death

play22:25

We need to focus the entire company

play22:27

on the AI circuit

play22:28

Because the key to this thing is that you have to be fast

play22:30

How fast?

play22:31

So fast that Google was about to twist its ankle

play22:36

We mentioned earlier that Microsoft’s press conference

play22:37

was on February 7th

play22:38

where they announced to integrate ChatGPT

play22:40

into their search engine

play22:41

Google in a hurry

play22:42

organised a press conference on February 8th

play22:43

and introduced a conversational AI called Bard

play22:45

This is developed based on their

play22:47

Chatbot LaMDA

play22:48

Just look at the stock prices of

play22:50

Microsoft and Google after Google's announcement

play22:52

You’d know how bad it is

play22:53

for Google

play22:56

You can't blame anyone in this business

play22:57

You don't have to read any professional analysis

play22:59

You just need to stay calm

play22:59

watch their press conference

play23:00

from beginning till the end

play23:01

then you’ll know why

play23:02

Everyone knows that

play23:03

people are focusing on

play23:04

AI chat

play23:05

But during Google's 40 minutes press conference

play23:07

they talked about their previous achievements

play23:09

and later picture search

play23:11

In the middle of this, the speaker

play23:13

couldn't find the mobile phone for presentation

play23:14

So have to skip this part

play23:21

Later, when they finally got to the point

play23:22

and begin to introduce Bard

play23:24

they only talked for a few minutes

play23:26

During the press conference

play23:27

they also showed a video

play23:28

introducing Bard

play23:30

The terrible thing is that there are

play23:31

factual mistakes in Bard's answer in the video

play23:35

Actually to be honest

play23:36

everyone can understand

play23:37

if this type of chatbot

play23:39

makes some factual error

play23:41

However

play23:41

the answer in the commercial video is incorrect

play23:43

and even forgot to bring mobile phone

play23:44

it was nothing but great cry and little wool

play23:46

It’s obvious that

play23:47

Google was doing it hastily

play23:49

this is what the market is worrying about

play23:52

Although ChatGPT is great

play23:53

but everyone knows that

play23:54

Google is the power house in AI sector

play23:56

So even though you didn’t

play23:57

make much noise

play23:58

the outsiders would know you not to be messed with

play24:00

and that you are holding back something big

play24:01

They initiated the red alert

play24:03

because it is to let

play24:04

outsiders know that they take this matter seriously

play24:07

Don’t sell out your stock first

play24:08

Before the press conference

play24:09

Google’s stock price is not worse than Microsoft

play24:11

But they had to hastily

play24:12

do something that

play24:13

make a fool of themselves

play24:15

So Google's market value evaporated

play24:17

$100 billion

play24:19

But in comparison, Microsoft is much more stable

play24:22

Microsoft’s CEO

play24:23

OpenAI’s CEO all came out

play24:24

and personally explain

play24:25

The nearly one-hour press conference

play24:26

focused on the AI chat function

play24:29

Plus various demos

play24:30

Obviously well prepared

play24:33

The AI war just started

play24:34

Google was first caught off guard

play24:36

by ChatGPT

play24:37

And then out of panic

play24:38

they did stupid mistake

play24:39

The first battle was a disastrous defeat.

play24:41

However this is only the first battle

play24:43

Google is still Google after all

play24:45

What’s gonna happen next

play24:46

we’ll just wait and see

play24:50

Of course this AI war

play24:51

is not limited to these two companies

play24:53

Meta, Baidu, Tencent, Ali

play24:55

are all fighting to get in

play24:56

Stocks that had anything to do with generative AI

play24:58

started to soar

play24:59

Nvidia, AMD

play25:00

hardware manufacturer that provides computing power foundation

play25:02

profited from this

play25:03

Actually for AI chatting, AI painting

play25:05

AI programming

play25:06

all these generative AI

play25:07

have been under development spurt

play25:09

since two years ago

play25:10

play25:11

The amount raised over the past few years

play25:12

From 2021, 2022

play25:13

It's already taking off

play25:14

It's over a billion dollars a year

play25:16

As the year 2023 begins

play25:18

Microsoft started with $10 billion

play25:20

Capital has done all it can

play25:21

to get into this track

play25:24

This thing is developing so fast

play25:26

will it cause many people to lose their jobs?

play25:27

Who will lose their jobs?

play25:29

Will it cost you your job?

play25:30

technological innovation

play25:32

It's always a double edged sword

play25:33

it may create more jobs

play25:35

Unemployment rate does not necessarily fall

play25:36

The overall GDP will probably rise

play25:38

But in the short term

play25:39

It will certainly cause some people to lose their jobs

play25:42

I was thinking how do you think we could

play25:44

try to not lose our job

play25:46

and even use this AI tool

play25:48

to increase our productivity

play25:49

My personal opinion

play25:51

is that we have to avoid

play25:52

repetitive routine work

play25:53

When computers first came out

play25:55

It might solve some

play25:56

repetitive human tasks

play25:58

Everyday doing the

play25:58

same thing over and over again

play26:00

You can fix it with a computer for loop

play26:02

But now

play26:03

it’s not just the repetitive works

play26:05

Even routine work

play26:06

as long as you have routine

play26:07

even though every day you think you are creating content

play26:09

but in actuality it doesn’t require much brain power

play26:11

then this kind of task the computer

play26:12

can quickly get it in the matter of minutes

play26:14

What is routine work?

play26:15

Let me give you an example

play26:16

For example I let ChatGPT

play26:18

write a fairy tales on Xiao Lin

play26:20

It’ll come up with Xiao lin has a cat that can speak

play26:23

it defeated dragon

play26:24

saved the princess and became a hero

play26:26

And I tell it, no it’s wrong

play26:26

Xiao Lin is a woman, rewrite it

play26:28

It’ll say Xiao lin is a woman

play26:29

who has a cat that can speak

play26:30

It defeated bad witch and became a hero

play26:32

You see

play26:32

this is the routine of a fairy tale

play26:34

There’s an animal that can speak

play26:35

defeated something and became a hero

play26:37

Although the speaking cat

play26:38

is basically useless in the story

play26:40

but it is the standard of fairy tales

play26:41

Similarly

play26:42

There are some exceptionally skilled engineers

play26:44

who can write codes

play26:44

with their eyes shut

play26:46

Writers who could

play26:47

write 20 chapters of online novel in a day

play26:48

Or some very basic

play26:49

financial report of the company

play26:50

basic design

play26:51

basic legal advice, etc

play26:54

For these tasks

play26:55

once you get familiar you can do it with eyes shut

play26:57

Because it has a routine

play26:58

So now AI is learning these routine

play27:00

so you don’t even have to do it anymore

play27:02

AI will do it all

play27:03

Attention, I’m not saying that

play27:04

programmer, accountant or writer

play27:06

or analyst will be replaced

play27:08

It’s just that the routine part of

play27:10

their work can be

play27:11

easily learned by machine

play27:13

So if you think there are some

play27:15

routine tasks in your job

play27:17

you need to be careful

play27:18

Or at least

play27:18

don’t put these routine work online

play27:20

Else AI will learn it

play27:23

Actually not just on unemployment

play27:25

Because it is so

play27:26

subversive

play27:27

we can already see

play27:28

the huge impact

play27:29

on society

play27:30

For example in education sector

play27:32

it's only been online for a few months

play27:33

Among the students

play27:34

over the age of 18 in the United States

play27:35

90% of them have used ChatGPT to help them with homework

play27:38

Apart from sports

play27:40

can it really help in any subject

play27:41

How would I know

play27:42

if you are doing the homework yourself

play27:43

Of course it doesn't mean that

play27:44

we can't use it to help

play27:45

It's just that our current education system

play27:47

is not ready for ChatGPT

play27:49

play27:50

It's as if we've spent hundreds of years

play27:52

building a better

play27:53

transportation system

play27:55

but suddenly one day

play27:55

all the cars could fly

play27:57

From technical view point, flying car

play27:58

is good in long-term

play27:59

but right now when we don’t have a

play28:01

complete new system

play28:02

while everyone is flying their car

play28:03

then it’ll be chaotic

play28:05

The social order

play28:05

would be greatly disrupted

play28:09

So for companies and schools

play28:10

they haven’t figure out how to integrate ChatGPT

play28:12

into their existing system

play28:13

For now they can only

play28:15

ban it first

play28:16

The content that AI created

play28:18

the painting it created

play28:20

who will own the copyright

play28:21

These are really tough questions

play28:23

So this generative AI

play28:24

no one can say for sure

play28:27

how it will develop in the future.

play28:27

The ChatGPT team

play28:29

did not have any special purpose

play28:30

at the very beginning

play28:32

Just put the data in

play28:33

and let the machine learns

play28:34

It was later that they discovered

play28:36

how powerful it is

play28:36

and could even connect to search engine

play28:38

Everyone is basically tapping in the dark

play28:40

You don’t know when one day

play28:42

the AI is suddenly become enlightened in one field

play28:44

Sometimes I feel that

play28:45

It is actually quite exciting

play28:48

to witness such a miraculous development of AI.

play28:49

Pandora's box is

play28:50

being opened bit by bit