Introduction to Generative AI

Summary

本视频课程介绍了生成性人工智能（Generative AI）的基础知识，包括定义、工作原理、模型类型和应用场景。课程解释了AI、机器学习和深度学习之间的关系，并探讨了生成性模型与判别性模型的区别。通过实例讲解了如何使用不同类型的生成性AI模型来生成文本、图像、音频等内容，以及如何通过提示设计来引导模型输出。此外，还介绍了Google Cloud提供的资源和工具，如Vertex AI Studio、Vertex AI Search and Conversation以及PaLM API，帮助开发者利用生成性AI解决实际问题。

Takeaways

• 📚 生成式AI是一种可以创造文本、图像、音频和合成数据的人工智能技术。
• 🤖 人工智能是计算机科学的一个分支，涉及创建能够自主推理、学习和行动的智能代理。
• 🧠 机器学习是AI的一个子领域，通过输入数据训练模型，使其能够对新的数据做出有用的预测。
• 🔍 监督学习模型使用带标签的数据，而无监督学习模型处理不带标签的数据。
• 🌟 深度学习是机器学习的一种，使用人工神经网络处理更复杂的模式。
• 🛠️ 生成式模型基于学习到的数据概率分布生成新数据实例，而判别式模型则用于分类或预测数据点的标签。
• 📈 生成式AI是深度学习的子集，使用人工神经网络，可以处理标记和未标记数据，并使用监督、无监督和半监督方法。
• 📝 生成式AI通过训练学习现有内容，创建统计模型，并在给定提示时生成新内容。
• 🔑 提示是给LLM的一小段文本输入，可以用来以多种方式控制模型的输出。
• 🏗️ 基础模型是在大量数据上预训练的大型AI模型，旨在适应（或微调）广泛的下游任务。
• 🌐 Vertex AI提供模型花园，其中包括用于聊天和文本的基础模型PaLM API和用于图像生成的稳定扩散模型。

Q & A

生成式AI是什么？

生成式AI是一种人工智能技术，能够生成各种类型的内容，包括文本、图像、音频和合成数据。

人工智能和机器学习有什么区别？

人工智能是计算机科学的一个分支，涉及创建能够自主推理、学习和行动的智能代理。机器学习是AI的一个子领域，是一种通过输入数据训练模型的程序或系统。

监督学习和无监督学习的主要区别是什么？

监督学习模型使用带有标签的数据，而无监督学习模型使用没有标签的数据。监督学习通过过去的示例来预测未来的值，而无监督学习关注于发现数据是否自然分组。

深度学习在机器学习方法中扮演什么角色？

深度学习是机器学习的一种类型，使用人工神经网络处理比传统机器学习更复杂的模式。人工神经网络受到人脑的启发，由许多相互连接的节点或神经元组成，可以通过处理数据和做出预测来学习执行任务。

生成式模型和判别式模型有何不同？

判别式模型用于对数据点进行分类或预测标签，而生成式模型基于学习到的现有数据的概率分布生成新数据实例。

如何判断一个输出是否是生成式AI的产物？

如果输出是自然语言、音频或图像等，那么它是生成式AI的产物。如果输出是一个数字、类别或概率，那么它不是生成式AI的产物。

生成式AI的过程与传统机器学习过程有何不同？

传统机器学习过程使用训练代码和标记数据来构建模型，而生成式AI过程可以使用训练代码、标记数据和未标记数据来构建一个基础模型，该模型能够生成新内容，如文本、代码、图像、音频、视频等。

什么是Transformers模型？

Transformers模型由编码器和解码器组成，编码器编码输入序列并将其传递给解码器，解码器学习如何解码表示以完成相关任务。Transformers在2018年引发了自然语言处理的革命。

什么是prompt设计？

Prompt设计是创建一个提示的过程，该提示将从大型语言模型（LLM）生成期望的输出。

基础模型是什么，它们如何帮助行业？

基础模型是大型AI模型，它们在大量数据上进行预训练，旨在适应（或微调）广泛的下游任务，如情感分析、图像描述和对象识别。基础模型有潜力革新许多行业，包括医疗保健、金融和客户服务。

Google Cloud如何帮助用户更多地利用生成式AI？

Google Cloud通过Vertex AI Studio、Vertex AI Search and Conversation以及PaLM API等工具，帮助用户探索、定制和部署生成式AI模型，即使没有编码或机器学习经验也能构建聊天机器人、数字助手、自定义搜索引擎等。

Gemini模型的特点是什么？

Gemini模型是一种多模态AI模型，它不仅能理解文本，还能分析图像、理解音频的细微差别，甚至解释编程代码。这使得Gemini能够执行以前对AI来说不可能的复杂任务。

Outlines

00:00

📚 介绍生成式人工智能

本段落介绍了生成式人工智能（Generative AI）的基本概念和应用领域。首先，通过定义生成式AI，解释了其作为人工智能技术的一个分支，能够生成包括文本、图像、音频和合成数据在内的各种类型的内容。接着，对人工智能（AI）进行了简要的背景介绍，区分了AI与机器学习（ML）的不同，并解释了AI是计算机科学的一个分支，涉及创建能够自主推理、学习和行动的智能代理。此外，还探讨了机器学习的两个主要类别：监督学习和无监督学习，并通过具体的例子说明了它们在实际问题中的应用。最后，引入了深度学习作为机器学习方法的一个子集，并强调了其在生成式AI中的重要性。

05:03

🧠 生成式与判别式模型

这一部分深入探讨了生成式AI在深度学习领域的位置，并解释了生成式模型与判别式模型的区别。生成式模型能够基于学习到的概率分布生成新的数据实例，而判别式模型则用于对数据点进行分类或预测标签。通过具体的例子，如狗和猫的图像分类，说明了两种模型的工作原理。此外，还讨论了生成式AI的识别方法，指出当输出是自然语言、音频或图像时，可以认为是生成式AI的应用。最后，通过数学视角，进一步阐述了生成式AI与传统机器学习过程的不同之处。

10:05

🌐 语言模型与内容生成

本段落聚焦于生成式AI在语言模型和内容生成方面的应用。首先，介绍了大型语言模型如何通过从互联网上摄取大量数据来构建基础语言模型，并通过提问来获取相关信息。接着，给出了生成式AI的正式定义，强调其通过学习现有内容来创造新内容的能力。此外，还讨论了不同类型的生成式AI模型，如生成式图像模型、生成式语言模型以及它们在文本、图像、音频和决策等方面的输出能力。最后，探讨了生成式语言模型作为模式匹配系统的工作方式，并通过具体例子展示了其在文本生成方面的强大功能。

15:08

🛠️ 应用与模型类型

这一部分详细介绍了生成式AI的各种应用和模型类型。首先，介绍了文本到文本、文本到图像、文本到视频和文本到3D等不同类型的模型，并解释了它们如何根据文本输入生成相应的输出。然后，讨论了文本到任务模型，它们能够根据文本输入执行定义好的任务或动作。接着，提出了基础模型的概念，这些大型AI模型在大量数据上预训练，并能够适应广泛的下游任务。此外，还提到了Google Cloud提供的Vertex AI和PaLM API等工具，以及它们如何帮助用户更有效地利用生成式AI。最后，通过代码生成的例子，展示了生成式AI在解决编程问题方面的潜力。

20:11

🌟 Google Cloud与生成式AI

本段落主要介绍了Google Cloud如何帮助用户更好地利用生成式AI。首先，介绍了Vertex AI Studio，这是一个允许用户快速探索和自定义生成式AI模型的工具，它提供了丰富的资源和工具，使开发者能够轻松入门。然后，提到了Vertex AI Search and Conversation，这是一个无需编码经验即可创建聊天机器人和数字助手的工具。最后，介绍了PaLM API，它允许开发者测试和实验Google的大型语言模型和生成式AI工具。此外，还提到了Model Garden，这是一个包含各种基础模型的库，这些模型可以用于情感分析、图像字幕、对象识别等多种任务。

Keywords

💡生成式人工智能

生成式人工智能（Generative AI）是一种能够基于已有内容生成新内容的人工智能技术，包括文本、图像、音频和合成数据等。在视频中，生成式AI是核心概念，它通过学习现有数据的模式和结构，生成新的、类似风格的样本。例如，生成式语言模型可以根据训练数据生成自然语言文本。

💡人工智能

人工智能（AI）是计算机科学的一个分支，涉及创建能够自主推理、学习和行动的智能代理系统。视频中提到，AI的目标是构建能够像人类一样思考和行动的机器，它包括理论和方法，使得机器能够模拟人类智能。

💡机器学习

机器学习是AI的一个子领域，指的是通过输入数据训练模型，使其能够对新的、未见过的数据做出有用的预测。在视频中，机器学习模型分为监督学习和非监督学习两种类型，它们通过不同方式处理数据，以实现预测和分类任务。

💡深度学习

深度学习是机器学习的一种技术，使用人工神经网络处理更复杂的模式。视频中提到，人工神经网络受到人脑的启发，由许多相互连接的节点或神经元组成，能够通过处理数据和做出预测来学习执行任务。深度学习模型通常具有多层神经元，使其能够学习比传统机器学习模型更复杂的模式。

💡生成模型

生成模型是一类能够基于学习到的概率分布生成新数据实例的模型。视频中解释说，生成模型不同于判别模型，后者用于对数据点进行分类或预测标签，而生成模型则生成新内容。例如，生成模型可以学习数据的联合概率分布，并预测出新的数据实例，如生成一张狗的图片。

💡判别模型

判别模型是用于对数据点进行分类或预测标签的模型。它们通常在标记数据集上进行训练，并学习数据点特征与标签之间的关系。视频中提到，一旦判别模型被训练，它就可以用来预测新数据点的标签。

💡半监督学习

半监督学习是一种结合了少量标记数据和大量未标记数据的神经网络训练方法。在视频中，半监督学习帮助神经网络学习任务的基本概念，同时通过大量未标记数据帮助网络泛化到新的例子。这种方法使得模型能够在有限的标记数据情况下，仍能有效学习和预测。

💡变换器（Transformers）

变换器是一种深度学习模型，它在自然语言处理领域引起了革命性的变化。视频中提到，变换器模型由编码器和解码器组成，编码器对输入序列进行编码，然后将编码后的表示传递给解码器，解码器学习如何针对相关任务解码这些表示。变换器在处理语言数据时非常有效，但也可能出现幻觉（hallucinations），即生成无意义或语法不正确的词语或短语。

💡幻觉（Hallucinations）

幻觉是指模型生成的无意义或语法不正确的词语或短语。在视频中，幻觉可能是由多种因素引起的，例如模型未在足够多的数据上进行训练、训练数据质量差、模型未获得足够的上下文或约束。幻觉可能会使输出文本难以理解，也可能导致模型生成错误或误导性信息。

💡提示（Prompts）

提示是提供给大型语言模型（LLM）的一小段文本输入，用于以多种方式控制模型的输出。在视频中，提示设计是指创建能够从大型语言模型生成期望输出的提示的过程。通过设计有效的提示，用户可以引导生成式AI产生特定类型的响应或内容。

💡基础模型（Foundation Models）

基础模型是大型AI模型，它们在大量数据上进行预训练，并被设计成能够适应（或微调）广泛的下游任务，如情感分析、图像描述和对象识别等。视频中提到，基础模型有潜力革新多个行业，包括医疗保健、金融和客户服务等。

💡代码生成

代码生成是指使用生成式AI技术自动创建编程代码的过程。在视频中，展示了如何通过输入代码文件转换问题，使用Gemini模型生成所需的步骤和代码片段。这种方法可以帮助开发者调试源代码、解释代码、编写SQL查询、将代码从一种语言翻译成另一种语言，以及生成源代码的文档和教程。

Highlights

生成性人工智能是一种可以创造各种类型内容的人工智能技术，包括文本、图像、音频和合成数据。

AI是计算机科学的一个分支，涉及创建能够推理、学习和自主行动的智能代理。

机器学习是AI的一个子领域，是一种通过输入数据训练模型的程序或系统。

监督学习模型使用带有标签的数据，而无监督学习模型使用没有标签的数据。

深度学习是使用人工神经网络的机器学习类型，能够处理比传统机器学习更复杂的模式。

生成性AI是深度学习的子集，使用人工神经网络，可以处理标记和未标记数据，使用监督、无监督和半监督方法。

生成性模型基于学习到的概率分布生成新数据实例，而判别性模型用于对数据点进行分类或预测标签。

传统编程需要硬编码规则，而生成性AI可以从训练数据中学习模式，并生成新内容。

生成性AI是一种根据从现有内容中学到的信息创造新内容的人工智能。

生成性语言模型可以学习语言中的模式，并通过文本输入生成更多文本、图像、音频或决策。

Transformer模型由编码器和解码器组成，引发了2018年自然语言处理的革命。

提示是给LLM的一小段文本输入，可以用来以多种方式控制模型的输出。

生成性AI模型类型包括文本到文本、文本到图像、文本到视频和文本到任务等。

基础模型有潜力革新多个行业，包括医疗保健、金融和客户服务等。

Google Cloud提供的生成性AI工具和服务包括Vertex AI、PaLM API和Gemini等，可以帮助开发者和企业利用生成性AI。

Transcripts

Hello, and welcome to “Introduction to Generative AI”.

Don't know what that is?

Then you're in the perfect place.

In this course, I'll teach you 4 things.

How to...

Define Generative AI

Explain how Generative AI works

Describe Generative AI Model Types

Describe Generative AI Applications

But let's not get swept away with all that yet.

Let's start by defining what Generative AI is first.

Generative AI has become a buzzword but what is it?

Generative AI is a type of artificial intelligence technology that can produce various types

of content- including text, imagery, audio and synthetic data.

But what is artificial intelligence?

Since we are going to explore Generative Artificial Intelligence, let’s provide a bit of context.

Two very common questions asked are:

What is artificial intelligence and what is the difference between AI and machine learning?

Let's get into it.

So one way to think about it is that AI is a discipline, like how physics is a discipline

of science.

AI is a branch of computer science that deals with the creation of intelligent agents, and

are systems that can reason, learn, and act autonomously.

Are you with me so far?

Essentially, AI has to do with the theory and methods to build machines that think and

act like humans.

Pretty simple right?

Now let's talk about machine learning.

Machine learning, is a subfield of AI.

It is a program or system that trains a model from input data.

The trained model can make useful predictions from new (never-before-seen) data drawn from

the same one used to train the model.

This means that Machine Learning gives the computer the ability to learn without explicit

programming.

So what do these Machine Learning models look like?

Two of the most common classes of machine learning models are unsupervised and supervised

ML models.

The key difference between the two is that with supervised models, we have labels.

Labeled data is data that comes with a tag, like a name, a type, or a number.

Unlabeled data is data that comes with no tag.

So what can you do with Supervised and unsupervised models?

This graph is an example of the sort of problem that a supervised model might try to solve.

For example, let’s say you are the owner of a restaurant, what type of food do they

serve?

Let's say pizza, wait dumplings, no pizza, I love pizza.

Anyway....

You have historical data of the bill amount and how much different people tipped based

on order type - pick-up or delivery.

In Supervised Learning, the model learns from past examples to predict future values.

Here, the model uses the total bill amount data to predict the future tip amount (based

on whether an order was picked-up or delivered).

Also guys... tip your delivery drivers, they work hard.

This is an example of the sort of problem that an unsupervised model might try to solve.

Here, you want to look at tenure and income, and then group or cluster employees, to see

whether someone is on the fast track.

Nice work blue shirt.

Unsupervised problems are all about discovery, about looking at the raw data, and seeing

if it naturally falls into groups.

This is a good start but let's go a little deeper to show this difference graphically

because understanding these concepts is the foundation for your understanding Generative

AI.

In supervised learning, testing data values (“x”) are input into the model.

The model outputs a prediction and compares it to the training data used to train the

model.

If the predicted test data values and actual training data values are far apart, that is

called error.

The model tries to reduce this error until the predicted and actual values are closer

together.

This is a classic optimization problem.

So let's check in.

So far we've explored the differences between artificial intelligence and machine learning

and supervised and unsupervised learning.

That's a good start!

But what's next?.

Let's briefly explore where deep learning fits as a subset of ML methods.

And then I promise we'll start talking about GenAI.

While machine learning is a broad field that encompasses many different techniques, deep

learning is a type of machine learning that uses artificial neural networks, allowing

them to process more complex patterns than machine learning.

Artificial neural networks are inspired by the human brain.

Pretty cool huh?

Like your brain, they are made up of many interconnected nodes, or neurons, that can

learn to perform tasks by processing data and making predictions.

Deep learning models typically have many layers of neurons, which allows them to learn more

complex patterns than traditional machine learning models.

Neural networks can use both labeled and unlabeled data.

This is called semi-supervised learning.

In semi-supervised learning, a neural network is trained on a small amount of labeled data

and a large amount of unlabeled data.

The labeled data helps the neural network to learn the basic concepts of the task, while

the unlabeled data helps the neural network to generalize to new examples.

Now we finally get to where Generative AI fits into this AI Discipline!

Gen AI is a subset of deep learning, which means it uses Artificial Neural Networks,

can process both labeled and unlabeled data, using supervised, unsupervised, and semi-supervised

methods.

LLMs are also a subset of Deep Learning.

See?

I told you I'd bring it all back to GenAI.

Good job me.

Deep learning models (or machine learning models in general) can be divided into two

types – generative and discriminative.

A discriminative model is a type of model that is used to classify or predict labels

for data points.

Discriminative models are typically trained on a dataset of labeled data points, and they

learn the relationship between the features of the data points and the labels.

Once a discriminative model is trained, it can be used to predict the label for new data

points.

A generative model generates new data instances based on a learned probability distribution

of existing data.

Generative models generate new content.

Take this example.

Here, the discriminative model ‌learns the conditional probability distribution or the

probability of “y” (our output) given “x” (our input), that this is a dog and

classifies it as a dog and not a cat, which is great because I'm allergic to cats.

The generative model ‌learns the joint probability distribution (or the probability of x and

y) p(x,y) and predicts the conditional probability that this is a dog and can then generate a

picture of a dog.

Good boy, I'm going to name him Fred.

To summarize:

Generative models can generate new data instances and

Discriminative models discriminate between different kinds of data instances.

One more quick example:

The top image shows a traditional machine learning model which attempts to learn the

relationship between the data and the label (or what you want to predict).

The bottom image shows a Generative AI Model which attempts to learn patterns on content

so that it can generate new content.

So what if someone challenges you to a game of "Is it GenAI or not?"

I've got your back.

This illustration shows a good way to distinguish between what is GenAI and what is not.

It is NOT GenAI when the output (or y” or label) is a number, or a class (for example

- Spam or not spam), or a probability.

It IS GenAI when the output is natural language (like speech or text), audio, or an image

like Fred from before, for example.

Let's get a little mathy to really show the difference.

Visualizing this mathematically would look like this.

If you haven't seen this for awhile, the Y=f(x) equation calculates the dependent output of

a process given different inputs.

The “Y” stands for the model output,, the “f” embodies the function used in

the calculation (or model), and the “X” represents the input or inputs used for the

formula.

As a reminder, inputs are the data like comma separated value files, text files, audio files

or image files like Fred So, the model output is a function of all the inputs.

If the “y” is a number - like predicted sales - it is not Generative AI.

If “y” is a sentence like “Define sales”, it is generative, as the question would elicit

a text response.

The response would be based on all the massive large data the model was already trained on.

So, the traditional ML Supervised Learning process takes training code and labeled data

to build a model.

Depending on the use case or problem, the model can give you a prediction, classify

something, or cluster something.

Now let's check out how much more robust the Generative AI process is in comparison.

The Generative AI process can take training code, labeled data, and unlabeled data of

all data types and build a “foundation model”.

The foundation model can then generate new content.

It can generate text, code, images, audio, video, and more.

We've come a long way from traditional programming, to neural networks, to generative models!

In traditional programming, we used to have to hard code the rules for distinguishing

a cat - type: animal,

legs: 4, ears: 2,

fur: yes, likes: yarn, catnip.

dislikes: Fred

In the wave of neural networks, we could give the network pictures of cats and dogs and

ask - “Is this a cat” - and it would predict a cat.... or not a cat.

What's really cool is that in the generative wave, we - as users - can generate our own

content - whether it be text, images, audio, video, or more.

For example, models like PaLM (or Pathways Language Model or LaMDA (or Language Model

for Dialogue Applications) ingest very, very large data from multiple sources across the

Internet and build foundation language models we can use simply by asking a question - whether

typing it into a prompt or verbally talking into the prompt itself.

So, when you ask it “what’s a cat”, it can give you everything it has learned

about a cat.

Now let's make things a little more formal with an official definition.

What is Generative AI?

GenAI is a type of Artificial Intelligence that creates new content based on what it

has learned from existing content.

The process of learning from existing content is called training and results in the creation

of a statistical model.

When given a prompt, GenAI uses this statistical model to predict what an expected response

might be–and this generates new content.

It learns the underlying structure of the data and can then generate new samples that

are similar to the data it was trained on.

Like I mentioned earlier, a generative language model can take what it has learned from the

examples it’s been shown and create something entirely new based on that information.

That's why we use the word “generative!”

But, large language models, which generate novel combinations of text in the form of

natural-sounding language, are only one type of generative AI .

A generative image model takes an image as input and can output text, another image,

or video.

For example, under the output text, you can get visual question and answering, while under

output image, an image completion is generated, and under output video, animation is generated.

A generative language model takes text as input and can output more text, an image,

audio, or decisions.

For example, under the output text, question and answering is generated, and under output

image a video is generated.

I mentioned We've stated that generative language models learn about patterns in language through

training data.

Then, give some text, then predict what comes next.

Thus, Generative language models are pattern-matching systems.

They learn about patterns based on the data you provide.

Check out this example:

Based on things it’s learned from its training data, it offers predictions of how to complete

this sentence.

“I'm making a sandwich with peanut butter and

__”Jelly.

Pretty simple right?

Here is the same example using Gemini, which is trained on a massive amount of text data,

and is able to communicate and generate human-like text in response to a wide range of prompts

and questions.

See how detailed the response can be?

Here is another example that's just a little more complicated that pb&j's.

The meaning of life is:

And even with a more ambiguous question Gemini gives you a contextual answer and then shows

the highest probability response.

The power of Generative AI comes from the use of Transformers.

Transformers produced the 2018 revolution in Natural Language Processing.

At a high-level, a Transformer model consists of an encoder and decoder.

The encoder encodes the input sequence and passes it to the decoder, which learns how

to decode the representations for a relevant task.

Sometimes Transformers runs into issues though.

In Transformers, Hallucinations are words or phrases that are generated by the model

that are often nonsensical or grammatically incorrect.

See, not great?

Hallucinations can be caused by a number of factors, like when:

The model is not trained on enough data,

The model is trained on noisy or dirty data,

The model is not given enough context, or

The model is not given enough constraints.

Hallucinations can be a problem for Transformers because they can make the output text difficult

to understand.

They can also make the model more likely to generate incorrect or misleading information.

So put simply...

hallucinations are bad.

Let's pivot slightly and talk about prompts.

A prompt is a short piece of text that is given to the LLM as input, and it can be used

to control the output of the model in a variety of ways.

Prompt design is the process of creating a prompt that will generate the desired output

from a large language model (LLM).

Like I mentioned earlier, Generative AI depends a lot on the training data that you have fed

into it.

It analyzes the patterns and structures of the input data, and thus “learns.”

But with access to a browser based prompt, you the user can generate your own content.

So let's talk a little bit about the models types available to us when text is our input,

and how they can be helpful in solving problems, like never being able to understand my friends

when they talk about soccer.

The first is...

Text-to-Text

Text-to-text models take a natural language input and produce text output.

These models are trained to learn the mapping between a pair of texts For example, translating

from one language to others.

Next we have Text-to-image

Text-to-image models are trained on a large set of images, each captioned with a short

text description.

Diffusion is one method used to achieve this.

There's also Text-to-video and Text-to-3D

Text-to-video models aim to generate a video representation from text input.

The input text can be anything from a single sentence to a full script, and the output

is a video that corresponds to the input text.

Similarly Text-to-3D models generate three-dimensional objects that correspond to a user’s text

description (for use in games or other 3D worlds).

and finally there's Text-to-task

Text-to-task models are trained to perform a defined task or action based on text input.

This task can be a wide range of actions such as answering a question, performing a search,

making a prediction, or taking some sort of action.

For example, a text-to-task model could be trained to navigate web UI or make changes

to a doc through the GUI.

See with these models I can actually understand why my friends are talking about when a game

is on.

Another model that's larger than those I mentioned is a foundation model, which is a large AI

model pre-trained on a vast quantity of data "designed to be adapted” (or fine-tuned)

to a wide range of downstream tasks, such as sentiment analysis, image captioning, and

object recognition.

Foundation models have the potential to revolutionize many industries, including healthcare, finance,

and customer service.

They can even be used to detect fraud and provide personalized customer support.

If you're looking for foundation models, Vertex AI offers a Model Garden that includes Foundation

Models.

The language Foundation Models include PaLM API for Chat and Text.

The Vision Foundation models includes stable diffusion, which has been shown to be effective

at generating high-quality images from text descriptions.

Let’s say you have a use case where you need to gather sentiments about how your customers

are feeling about your product or service, you can use the classification task sentiment

analysis task model.

Same for vision tasks - if you need to perform occupancy analytics, there is a task specific

model for your use case.

So those are some examples of foundation models we can use, but can GenAI help with code for

your apps?

Absolutely.

Shown here are Generative AI Applications.

You can see there's quite a lot.

Let’s look at an example of code generation shown in the second block under code at the

top.

In this example, I’ve input a code file conversion problem - converting from Python

to JSON.

I use Gemini and insert into the prompt box “I have a Pandas DataFrame with two columns

– one with the filename and one with the hour in which it is generated.

I am trying to convert it into a JSON file in the format shown on screen.

Gemini returns the steps I need to do this – and the code snippet!

And here my output is in a JSON format.

Pretty cool huh?

Well get ready, it gets even better.

It gets better.

I happen to be using Google’s free-browser based Jupyter Notebook and can simple export

the Python code to Google’s Colab.

So to summarize - Gemini Code Generation Can Help You….

Debug your lines of source code.

Explain your code to you line by line.

Craft SQL queries for your database.

Translate code from one language to another.

Generate documentation and tutorials for source code.

I'm going to tell you about three other ways Google Cloud can help you get more out of

generative AI.

The First is Vertex AI Studio Vertex AI Studio lets you quickly explore

and customize generative AI models that you can leverage in your applications on Google

Cloud.

Vertex AI Studio helps developers create and deploy generative AI models by providing a

variety of tools and resources that make it easy to get started.

For example, there is a: Library of pre-trained models

Tool for fine-tuning models Tool for deploying models to production

Community forum for developers to share ideas and collaborate

Next we have Vertex AI which is particularly helpful for all of you who don't have much

coding experience...

You can build generative AI search and conversations for customers and employees with Vertex AI

Search and Conversation (formerly Gen AI App Builder).

Build with little or no coding and no prior machine learning experience.

Vertex AI can help you can create your own:

Chatbots Digital assistants

Custom search engines Knowledge bases

Training applications And more

And then we have PaLM API.

PaLM API let’s you test and experiment with Google’s Large Language Models and Gen AI

tools.

To make prototyping quick and more accessible, developers can integrate PaLM API with MakerSuite

and use it to access the API using a graphical user interface.

The suite includes a number of different tools, such as a model training tool, a model deployment

tool, and a model monitoring tool.

... and what do these tools do?

I'm so glad you asked.

The model training tool helps developers train ML models on their data using different algorithms.

The model deployment tool helps developers deploy ML models to production with a number

of different deployment options.

The model monitoring tool helps developers monitor the performance of their ML models

in production using a dashboard and a number of different metrics.

Lastly, there is Gemini, a multimodal AI model.

Unlike traditional language models, it's not limited to understanding text alone.

It can analyze images, understand the nuances of audio, and even interpret programming code.

This allows Gemini to perform complex tasks that were previously impossible for AI.

Due to its advanced architecture, Gemini is incredibly adaptable and scalable, making

it suitable for diverse applications.

Model Garden is continuously updated to include new models

And now you know absolutely everything about Generative AI... okay maybe you don't know

everything, but you definitely know the basics!

Thank you for watching our course and make sure to check out our other videos if you

want to learn more about how you can use AI!