Unpopular Opinion: LLMs Didn't Add Value to AIs—They Just Made Them More Accessible
Summary
TLDRこのトークは、LMS(Language Models)が技術的に新しいテーブルを持ち出しているか、それとも単に技術をよりアクセス可能にし、利用を拡大したにすぎないかという議論の中心に立脚しています。パネルは、LMSがビジネスや一般大衆の理解にどのように影響を与えたか、そしてこれらのモデルが将来どのように進化し、どのよう活用されるかについて洞察を提供します。彼らは、LMSが特定の業界でどのように使用され、その利点と課題をどのようにバランスを取るかについても議論します。
Takeaways
- 🤖 AIは、人間のように行動するコンピュータシステムの総称であり、特に機械学習がその大きな分野の中で中心的な役割を果たしています。
- 📊 LMS(Language Models)は技術的には新しいものではなく、RNNやCNNなどの既存技術に基つていますが、より高いアクセス性と広範な利用をもたらしました。
- 🚀 Transformerアーキテクチャは、より大きなテキストや文脈を処理できるようになり、自然言語処理の分野で大きな進歩を遂げました。
- 🧠 LMSの進化は技術的な進歩だけでなく、ビジネス価値の発見と適用面での革新も重要です。
- 💡 LMSは現在、単純なタスクを自動化し、生産性を向上させることができていますが、複雑な意思決定やビジネスプロセスを完全に takeover することはまだ達成されていません。
- 📈 LMSの成功事例は、医療や法律などのテキストに基づく分野で特に高いポテンシャルがあり、専門用語や規則に基づいたタスクを効率的に処理できるようになりました。
- 🔍 LMSの信頼性については、まだ完璧ではありません。実際の専門家と比較して、LMSは試験で高いスコアを獲得しても、実際の業務では不十分な場合があります。
- 🌐 LMSが将来的にシステムとの最初の接触点となる可能性があるとの視点があるが、効率的なユーザーインタラクションを実現するために、様々な方法が検討され続けています。
- 🔗 LMSの発展は、多様な言語や文化を考慮に入れた包括的なアプローチが求められ、これにより技術の普及と利用が促進されます。
- 📚 LMSの今後の進化は、応答の正確さと信頼性を向上させることで、より多くのビジネス分野で実装され、より広範に受け入れられるようになるでしょう。
Q & A
LMSが既存のAIシステムにどのような価値を提供したと思いますか?
-LMSは、技術的には新しい価値を提供していないものの、よりアクセスしやすいインターフェースを提供し、既存のAIシステムをより广く利用できるようにしました。
LMSが技術的にどの分野で進化を遂げたと感じますか?
-LMSは、Transformersアーキテクチャを基にしており、より大きなテキストデータセットを扱えるようになり、自然言語処理のコンテキストを大幅に改善しました。
LMSがビジネス価値を提供するための課題は何だと思いますか?
-LMSがビジネス価値を提供するためには、技術的な課題だけでなく、適用層でのイノベーションも重要です。特定のビジネスニーズに合ったLMSの活用方法を見つけることが鍵となります。
LMSが医療や法務などの規制された分野でどのように役立つでしょうか?
-LMSは、医療や法務などの分野で、豊富なテキストデータの処理やルールベースの分析を行い、専門家の仕事をサポートすることができます。ただし、LMSは最終的な決定を下すわけではなく、専門家の判断を補完する役割を果たします。
LMSの応用において、どのような技術的な課題があると感じますか?
-LMSの応用においては、技術的な正確性と信頼性の向上、そして実際の現場での適用可能性の確認が課題となっています。また、LMSが誤った情報を提供するリスクにも注意が必要です。
LMSが自然言語処理の分野でどのような進化を見せましたか?
-LMSは自然言語処理の分野で、より長いテキストや複雑な文脈を理解し、応答する能力を向上させました。これにより、より自然な人机交互が実現し、さまざまな業界で応用が可能となりました。
LMSを導入する際、どのようなビジネスのニーズを考慮すべきですか?
-LMSを導入する際には、ビジネスプロセスの効率化、カスタマーサービスの改善、データ分析の支援など、企業が抱える課題やニーズに応じた適切な活用方法を検討する必要があります。
LMSが提供する自然言語処理能力を最大限に活用するためには、どのようなアプローチが効果的ですか?
-LMSを効果的に活用するためには、適切なデータセットで事前学習を行い、特定のタスクや業界に関連するデータをファインチューンすることが重要です。また、ユーザーとのインタラクションを通じてLMSを徐々に改善していくことも効果的です。
LMSの開発において、今後どのような進化が期待されています?
-LMSの開発において、今後期待される進化には、より高精度な自然言語理解能力の向上、多様なデータタイプとの連携能力の強化、そして倫理的な問題に対する対処などが含まれます。
LMSが誤った情報を提供するリスクがあることはどのように対処すべきですか?
-LMSが誤った情報を提供するリスクに対しては、適切なデータセットで学習させ、定期的な評価とファインチューニングを行い、また、重要な決定においては人間の審査や介入を確保することが重要です。
LMSを導入する際のユーザーエクスペリエンスの改善について、どのようなポイントが重要ですか?
-LMSを導入する際には、ユーザーが自然で使いやすいインターフェースを提供し、不明瞭なプロンプトや質問を最小限に抑えることが重要です。また、フィードバックメカニズムを設けることで、ユーザーの声を取り入れながらLMSを改善していくことも効果的です。
Outlines
🎤 イベントの開会とLMSの議論
イベントの開会とLMSの価値についての議論が行われています。議論の中心は、LMSが既存のAIシステムに新たな価値を提供したのか、単に技術をよりアクセスしやすくしたのかということです。また、LMSが技術的な進歩をもたらしたのか、または単にインターフェースとして機能したかについても話し及んでいます。
🌟 LMSの進化と技術的な革新
LMSの進化と技術的な革新について、3人のパネルリストが見解を共有しています。彼らはLMSが持続的な技術の進歩であると思わないという意見も示しており、LMSが現在注目されている理由は、技術的な進歩だけでなく、その技術が一般大衆にアクセス可能になったことにあると考えています。また、トランスフォーマーアーキテクチャとその前身の技術がどのように進化してきましたか、そしてLMSがどのようにスケールとコンテキストを扱う能力を向上させたかについても議論されています。
💡 LMSのビジネス価値と産業への影響
LMSがビジネスと産業に与える影響について、パネルリストが意見を述べています。LMSは企業にとって新しいビジネスケースを開き、AI技術をより使いやすくする重要な要素となりました。また、LMSは特定の業界(医療や法律など)において、ルールベースの分野でのパターン認識を改善し、効率を向上させる可能性があると指摘されています。
🚀 LMSの課題と今後の方向性
LMSの現在の課題と今後の方向性について、パネルリストが議論しています。LMSの応用が広まる中で、重要なのはモデルの正当性と信頼性の確保です。LMSは確率的なモデルであり、実際には期待通りに機能しない場合もあります。そのため、LMSの適切な訓練、フィードバック、改善が重要です。また、LMSが単に応答を生成するだけでなく、ユーザーとインタラクションを行って学び、改善する双向的なシステムとなることが期待されています。
🌐 LMSの多様性と言語の役割
LMSの多様性と言語の役割について、パネルリストが見解を共有しています。LMSは多様なモダリティ(テキスト、画像、動画など)を扱うことができ、将来的には基盤としてさらに発展する可能性があるとされています。また、言語は人間の間だけでなく、機械間のインターフェースとしても役立つとされ、LMSは自然言語でのインタラクションを容易にするでしょう。
📈 LMSの成功と信頼性のしやすさ
LMSの成功と信頼性のしやすさについて、パネルリストが意見を述べています。LMSは完璧ではないため、適切な成功率を設定することが重要です。簡単なタスクではLMSが自動化できるものの、複雑な決定には人間の参加が必要であるとされています。また、LMSは応答を生成するだけでなく、ユーザーのフィードバックを学び、改善することが望ましい双向的な学習システムとなるべきです。
Mindmap
Keywords
💡LMS (Language Models)
💡AI (Artificial Intelligence)
💡Transformers
💡Fine-tuning
💡Accessibility
💡Probabilistic models
💡Human in the loop
💡Semantic understanding
💡Stochastic parrots
💡Multimodality
💡Knowledge base
Highlights
The panel discussion revolves around the value addition of Language Models (LMs) and their impact on Artificial Intelligence (AI) systems.
The panelists include experts from tech consulting, machine learning project management, and Google Cloud, providing diverse perspectives on LMs.
The discussion explores whether LMs have brought new technical advancements or simply increased accessibility to existing AI technologies.
Panelists share their definitions of AI, highlighting the importance of machine learning and the ability of systems to mimic human intelligence.
The evolution of AI is described as a continuum, with LMs like ChatGPT and DALL-E relying on transformer architectures and previous technological advancements.
The accessibility factor of LMs is emphasized, as they have made AI more approachable and fun for the general public.
The role of LMs in transforming user interaction with systems is discussed, with a focus on their potential as the first point of contact in various industries.
The importance of fine-tuning LMs for specific industry use cases, such as medical and legal applications, is highlighted to improve their accuracy and relevance.
The challenge of validating LMs is addressed, as their probabilistic nature makes it difficult to ensure their reliability in practical applications.
The potential of LMs to assist human experts in decision-making processes is discussed, with the emphasis on their current role as supportive tools rather than replacements.
The discussion touches on the need for LMs to understand context and handle complex queries, such as distinguishing between different types of quarters in a financial query.
The future of LMs is envisioned as part of the infrastructure, integrating with various data types and systems to form a comprehensive knowledge base.
The importance of inclusivity in LM interactions is stressed, with the need for systems to cater to diverse language abilities and literacy levels.
The potential of LMs to learn from human feedback, gradually improving their performance and ability to automate tasks, is highlighted as a promising area of development.
The debate on LMs as 'stochastic parrots' is addressed, acknowledging their probabilistic nature while also considering their potential for reasoning and understanding.
The discussion concludes with thoughts on the threshold of success rate for LM implementation and the need for a human-in-the-loop for sensitive use cases.
Transcripts
okay welcome uh to our panel the second
but last talk um on you can see the
topic on public opinion LMS didn't add
any value to our existing AIS um they
just made more accessible what we
already had and uh when I was scoping
the topic two or three months ago I
couldn't imagine that we're g to have
that many talks on LMS uh but here we
are and uh even if you have attended
former talks I'm sure you're going to
find this one uh valuable as well so I'm
chrisan I joined project a this year in
the tech Consulting
team and I'm super curious to find out
what our panelists have to say um I
already read out the topic um we're
going to talk about um if LMS brought
anything to new to the table technically
um or if they were just like acting as
an interface or a font technology and
making thereby LMS more accessible So
today we're going to have the goal of um
giving you some context to better
understand the the real technical
advancement behind behind LMS and also
to give you some food for thought um how
LMS could develop further and um how
they could be utilized in a way that
they provide real undeniable value with
me today are three lovely panelists um
we all like you all had have paneled in
different um
situations recently at the I think J
breakfast for marantic so um to start
off Hannah Hannah is um the team lead
for machine learning project management
at metics momentum marantic momentum for
those of you who don't know um
identifies use cases for II and ml um
with their customers and also then
develops and um um also operates those
um those solutions to to the use cases
then yakob you have seen yakob on stage
this morning already um yakob is a
customer engineer here at Google cloud
and uh is basically together with their
clients developing and um optimizing the
data infrastructure so um for all Cloud
native gcp clients um I would say I
would call you a an ML and II Enthusiast
and you're appearing like recently
appearing on many talks panels
hackathons um yeah and last but not
least is my dear colleague saman who's a
data scientist and engineer here project
a um together with our ventes he works
on data infrastructure machine learning
and also uh data
culture so um yeah I'm super happy to
have you here and um to kick things off
I'd like to uh get some interaction and
um ask you the audience on your opinion
um which on which side um you see
yourself also on which side on the
Spectrum so I would like to raise your
hand if you you are of the opinion that
LMS didn't add any value to our existing
a
technically is there anyone who's of the
opinion that llms didn't add any
value okay counter check if you're just
lazy raising your hand or because no one
is of the opinion who's of the opinion
that they add any that they did add any
value technically
okay so then we I think we're going to
have a interesting discussion um and to
kick things off and go right in I would
like to ask you three um to give us a
one- sentence definition of AI because
of course we know everyone defines a
differently uh and then so we can can
put your opinion um into perspective
maybe to start off with Hannah
sure no it was on before it was on
before sorry yeah happy to kick it off
so for me AI is really a collective term
for any kind of computer systems that in
some way simulate
yeah Behavior that's otherwise
identified or associated with human
behavior or intelligent behavior and I
think for me very pragmatically most
importantly is really this machine learn
topic of machine learning um as the
biggest cluster within the huge field of
AI yeah pretty I can I'm pretty much in
line with what he said but maybe to put
it into one sentence I think AI is a
collection of statistical and
mathematical
that sort of mimic human decision making
in some
sense I'm also in line with all the
other panels have said but I think I
would just go a bit broader and move
away from the statistical part of it and
just say any system that in one way or
the other um mimics aspect of human
intelligence whether that might be
Vision speech text writing that I was
ciz as as
AI Co thanks a lot so um to let's get uh
so let's get technical for a minute um
at project a we value knowledge sharing
really a lot so that's why for example
our conference tomorrow is called
project a knowledge conference and also
um at project a we have regular brown
bag lunches uh where a or colleagues can
share recent project outcomes or also
like any topic they find relevant to
them or they care of and recently saman
shared or did a brownb lunch on AI in
general and had a pretty I would say
critical view on this um beyond the bus
I would say and that's why I like to ask
you um is the Advent of LMS rather a
Continuum or a prival technical
advancement um in the ai's journey and
why or why
not um I think technically I wouldn't
say it's pivotal actually um I think if
you look at it from a technical
background current implementation of AI
which I think today we talk about these
geni aspects like chat GPT um Dar ma
Journey what else you have um rely on
this transform architecture which
basically then relies on lsdm which then
relies on RNN CNN so I think you have a
clear Evolution and and history of you
might even say incremental additions and
changes that have let to this point and
therefore arguing that now suddenly we
have these these these gen
uh applications are suddenly changing
would be a bit well exaggerating I would
say and I think if there was something
like shim who were in here he would now
say oh no I did all of this and I think
I would maybe partly agree with this as
on a technical level it's a clear
Evolution what
however makes us such a buzz in my
opinion that's why we have all of these
panels and we talk about this so much is
then um that somebody like my 67y old
dad now has a chat GPT app on his phone
and uses his because it's funny and cool
and that's essentially I think the
accessibility factor of it that previous
implementations were not able to do if
you talk to somebody about an lsdm is
they're like bro I don't know
but then you can show them hey give me a
cooking list for whatever I have in my
fridge and they can use it and they're
like oh this is insane and this
is a bit where I would sort of of argue
it's definitely not a pivotal change
technically um might be pivotal in sense
of how it's being perceived and
applied dig a bit into deeper into this
um what especially so you mentioned for
example Transformer architecture um is
this is this what you would you would
consider the technical advancement for
example that you now can um really
process it in parallel or can um get can
input like larger contents um or
contexts and it can understand it or
where do you see the technical
advancement because otherwise I would
say um there is there is nothing that
that we can really like um yeah hold on
to that that says okay it's technical
technically um I suppose and and I think
that's also so funny when we talk about
Transformers because they came out in
2017 I think was your former co-workers
paper when it was published um uh the
the main benefit I would say of
Transformers compared to say current
applications or for applications was
then you were able to um which goes into
this paration effort is you are now able
to work with much much much larger
corpuses Cory of body that were not able
to do so before right um if you take a
look at again it's lsdm which we might
have had before you were may be able to
go back as far as a sentence or two
because then you've got into these
diminishing returns and you're like oh I
don't even remember what you said and
now you know chat GPT can write you
books of content because this sort of
shortcoming has been alleviated with SC
implementation which yeah can be
attributed to a Transformer sure um
applications of it then um of what say
open AI does what BART does I think are
then more that on
scale maybe to add on to that I I
absolutely agree with what you've said
um I think what's I think what's very
much a Continuum is also when you think
about language we are always or the the
whole field of natural language
processing is kind of concerned about
capturing context and for the longest
time this was incredibly hard um as you
said like we would didn't really get
past sentences of context that were able
to be processed and then 2017 this
Transformer comes along of course also
building on Tech that was there
previously but this basically sparked
this explosion of yeah exponential
growth in what you can process in terms
of context whole whole books whole like
even libraries maybe at this stage so um
this this is really incredible and I
would also agree that the change we're
seeing now what's quite pivotal is um
definitely that we can now as an end
user also shape more how tool behaves um
without having to train it without
having to be super technical about it
but I do think this is partially also a
technical um an invention for for sure
um coming from the way these models are
pre- chained and now allow us to for
example um yeah give examples give
context provide intense and say dear
model please behave as if you were XY Z
or please write in the style of this um
and and this is quite quite pivotal
because this definitely makes some
things way easier and we can discuss
later if this is uh if this this a
Continuum or if this is really
groundbreaking yeah I will come back to
this definitely um yob why why did your
colleagues um did take so much time from
2017 until now um until they until
transform architecture really um yeah
really developed itself to to a point
where could could be utilized in a way
that that we can now really um see at
least a business value of of
course hard for me to make a statement
for for the researchers at this point
from from my perspective obviously but
yeah interesting question I think it's
really in line with the fact that in my
opinion technology usually doesn't make
pivotal changes and that's sometimes a
little bit of a tough truth because I
think us as humans we're also a little
bit wired in the sense that we love we
love the story aspect about these
individual Geniuses that make an
invention and then everything changes
and then we have suddenly this blackbox
solution that just solves all our
problems based on one genius's idea but
usually usually in technology and
research it doesn't work like that it's
rather one researcher Builds on the work
of many others we're all sort of working
on shoulders of giants in the end we
never invent everything from scratch and
that's also fine like that right that's
that really Mak the research uh culture
and the research Community right that
they collaborate and build on top of
each other's ideas and really the
pivotal changes in my opinion happen on
the application layer similar to what
someone said right suddenly there's a
break breakthrough someone um had the
idea of hey why don't we bring together
this technology with a certain business
use case where we see a large large need
this could solve this specific use case
and then these two perspectives come
together and when this B both both fits
right there's a technology that has
certain strength and then on the other
hand you have a business use case that
can be solved um and that makes sense
together and if this actually has value
then it really explodes then you have
your P to change but again that's more
rather on the application layer than on
a tech
technological in my opinion yeah so it's
it's partly a interface because it's
sticking together to Technologies and
then there would be an interface um like
the application acting as an interface
and making
front end whatever you want to call it
and making it then
accessible in a sense yeah of course of
course of course I mean in the end you
pluck together multiple Technologies and
then you try to get certain output right
yeah absolutely okay and then um to to
briefly talk about like the business
value um of llms Hannah you work closely
together with clients across Industries
um what would what would you say um the
introduction of llms um has it or like
where has it led to a significant change
in the perception but also in the
understanding um a among the masses and
B um in in business
General yeah so I think and I I would
expect we all agree with that and you
already mentioned that what what's
really this amazing Trend we are seeing
is that um everyone is getting hands on
and trying to work with with these tools
so I don't really know anyone who hasn't
at least tried to generate some greeting
card text or something like that with
llms which is amazing to see but to to
talk about industry I think maybe two
interesting things happen so first of
all we can we can feel the formo uh so
if there was anyone before not not
thinking about okay how can I use these
these how can I use AI essentially to
accelerate my bris business is now for
sure thinking about it at least if not
acting on it and and secondly um of
course it also unlocks certain use cases
or certain thoughts and I would also say
of course sometimes the the
conversations we have it seems like chbt
is is seen as like the solution to
everything which is clearly not the case
and we are also seeing that for example
in Industry we we have a lot of clients
in Industry um the main data sources
that we really have there are time
series and tabular data and this should
not be underestimated so everything that
was always said before like build up
your data um warehouses build up your
lake houses build up your um make sure
you you build the Foundation to utilize
this value this still holds true um up
to this point and there are a ton of
methods that should not be
underestimated um and then on top of
that language comes in as a very
interesting interface to access this
data in a new way and I think that's
that's really cool to see um but we also
we always have to make a bit of this
trade-off to see where is really where
is it language that solves the use case
and where is language that's the cool
addition to have a new way to interact
can I just react on this so generally
100% get your get your basic straight
before you jump into the more Advan St
stuff couldn't agree more but where I
would disagree with you is really since
we in the point you said um that you see
most with customers you see mostly
structured data in Time series yes they
do have a lot of that but they have a
lot of that because they didn't pay
attention to all the textual data or
potentially the data that could
potentially Ed be used for um for NLP
use cases I would say so obviously
there's a lot of structured data and
time series and obviously companies
still struggle with the basics in
getting really the infrastructure state
but um I would say there is even more
textual data they just didn't unlock
that yet as data that has potential to
be used for AI so much absolutely yeah I
think I can I can only agree with that
and I think maybe to to add one one
aspect there um um is that I think in
every company us have functional units
that have a lot of text being that the
legal space being that um yeah HR
internal knowledge bases way where you
have a lot of data that you can unlock
um in in new ways um nowadays and then
in your core business it really depends
on what you're doing if that's also
textual or if that's um different data
but I agree um in that sense yes I think
every company has to some extent
potential to look at your text
Data thanks a lot for for for the rewind
um to come to like the current current
scenario um we briefly talked Hana you
mentioned it um about like the the the
way you WR prompts um also there's F
short learning um you mentioned it
before in your talk in your talk as well
to to um utilize for example giving a
demonstration of um of different use
case of um the things that the AL is
supposed to do um to like fine tune it
in a way
um is there um any are there any
examples could you share any example
where where for example these
um the these two types of fine-tuning in
LM a task osting llm um has led to any
impact in any
industry um by for example adding this
context and thereby for example
unlocking longtail use cases that
formerly couldn't couldn't even Sol
because there's simply no return invest
sure so generally what I'm seeing in in
my experience from working with LMS is
that find tuning multi things so
generally supervised learning in the
sense what we already know fromal
machine learning makes a lot of sense
and it increases the quality of your
answers by a huge margin um so I believe
most use cases will go in the direction
of using some sort of multi pumping fine
tuning Etc um that's the basic basic
layer to as as an answer of your
question in terms of specific industry
use cases yes I think there are some
industries that have high potential for
the use of LMS for example Med medical
industry legal use cases um some others
right um but you can already see that
because these are very text based at the
same time text based usually um but at
the same time very rule based in some
sense right it's sort of pattern
recognition in both both of these cases
a lot of cases but at the same time
they're also highly regulated and you
work with highly sensitive data and
specifically in the medical case you
work with data and you work with
decisions that you should not get
wrong right um even more in the medical
space because it's sometimes about lives
right in legal space it's a lot it's
about a lot of money usually so there is
an inherit value for llms the task is um
makes sense to solve but we have
specific challenges why anms need to get
these points right and in these cases we
probably should think about how to train
L&M that are specific for the industry
great example for this is met Palm so
our foundation model is Palm right and
we're one of the research projects that
is in preview at the moment actually is
metp for example right our research
trained Palm fine tuned it on medical
data um fine tun it for this specific
use case so essentially in the back
combine factual knowledge that the llm
needs to get right in any case with the
generational knowledge of of Pama the
generational capabilities right and this
works quite well I mean p is not public
yet but it already outperforms any other
model in the medical um certification
tests in the US for example um similar
for the legal knowledge I don't I'm not
aware at least for legal llm purely
legal find you LM probably it's out
there already but I can imagine very
well that will fine tune or train
specifically in llm on for example a
certain legal codex um to to answer
specific questions and compare claims
between parties um and have the legal
system in the rules of that yeah thanks
so some we at project a and and you and
your work we we don't have the capacity
to train whole models ourselves to find
you find you them um have you seen any
recent either in our portfolio or like
um on on blogs or have you read about
recent implementation um success stories
also challenges uh with
llms um
I think I would go in the direction that
that that yakob just mentioned um
because smart struggle with LMS I don't
want to talk too much of the skeptic is
in this aspect um there is a possibility
to find trun LM based on these
foundational models given a certain
distinct situation or data set and make
them perform very very well on those you
mentioned Health you mentioned legal HR
possible whatever um however what I then
very much strug given that these are
strongly probabilistic models is the
possibility
that these actually are not doing what
they're supposed to do because um
validating these llms is incredibly
difficult um so there are numerous
articles where I've seen that while they
might perform very very well on these um
legal bar exams or these Health exams
whatever that might be putting them to
practice they always are being
outperformed easily by an actual lawyer
by an actual doctor um which makes this
this um intelligence aspect which is
being touted as this the strength of
these llms kind of ambiguous because
what does it actually mean if you
perform well on the test but then in
practice you're actually quite shite um
I don't know I'm not the one to answer
this question but this is why I sort of
struggle with this notion that when you
have some sort of AI model in an LM
model it's opposed to um I don't want to
say replace but it's actually I think
just there's an assistant and therefore
I I I'm bit skeptical to see it's going
to have this massive massive impact that
a much much simpler models have had so
far so um if if you ask me if there's
one thing that is the biggest or had the
biggest value to anything it would be
linear regression but this came out what
150 years ago or something right U but
it still works and it works very
very well um and if LM I think there's
this risk to it that that might not be a
more deterministic approach that that
integration might offer
um so I think for me and and I think
it's going bit divert a bit from your
original question um thinking of
industry and application my struggle
still seems to be a validation and
making sure that these things work the
way they're supposed to do no as on like
old man get get clout but this is sort
of the approach that I take when we
think about applying these things to
Industries because no in our portfolio
it's not really that big um because
there I think we rely also much more on
hanus approach mentioned earlier like
get your Basics right get your linear
regression right before you can talk
about um llms etc etc etc because that's
too far away and most of the time might
not have impact you actually desire
might you add on this I would love to I
think I would like to share one example
which I think illustrates quite well how
in this case the legal space Also
benefits from the recent advantages and
also how we at momentum kind of handle
this this risk because yes I absolutely
agree um we have probabilistic models
here we have models that hallucinate um
we have these risks and it's very
important for us to make that very very
transparent and handle that but I do
think that assisting work um while an
expert a human expert has the final say
is still something that can have huge
benefits um huge impact so one case i'
I'd like to share from our project um
portfolio is that we are working with a
with a legal partner um and this one is
about um yeah working and processing
relatively standardized
um legal regulation um think basically
ndas or something like that um and
what's very interesting is that I think
handling something in the legal space
benefits a lot from recent advantages so
one thing for sure is models being
fine-tuned on legal data in general so
being able to process this kind of
language and being able to process
longer documents with a lot of context
but I what I find very interesting and
this wasn't basically possible um 10
months ago is that in the legal space
you also always have not only one fixed
set of laws but you also have a
continuous flow of cases of reference
cases you need to consider if you're
making a specific decision and um this
new way of context specific learning
essentially allows you to feed that into
your model without having to train it
but just you can give examples you can
give reference cases and you can extract
Knowledge from all of that and at least
like provide a summary of um what the
models then train to um detect as
relevant passages um give that to an
expert and it's already helping you and
giving these pointers and giving
references and yes it needs a lot of
constraint so that it will give this
references but it allows you to combine
internal and external sources and I
think that's very very um cool to see
and definitely something that's that's
possible and that we are currently
working on for example I wouldn't AG I
wouldn't disagree with this I think I
would actually sort of agree with this
um because I think the main benefit of
this as you said is assistance that I
would say complement existing work so
you always read about um companies that
use AI had an productivity benefit of
uplift of what 20% and I think that's
that 20% that a lawyer would spend
reading this long boring documents for
being P to make sense of what's going on
there and and that's essentially when we
go to a bit more of an established
approach which is sematic understanding
right I think translation of documents
was more less a solved issue in in 2010
whatever right and and there we had a
clear Benchmark right um I don't know
what it's called but it's essentially
when you have a machine translated text
versus human translated text how Sim
arties to and that essentially defines
how well the transation is going and
that's how we also measured how well
Transformers perform actually whenever
launched um I think this you this this
one Benchmark is the what's missing to
then rely on what's a legal um what say
illegal AI might do and I ask a gen
based on legal text hey is this bad am I
being suit now I wouldn't trust it
advice but I would trust it hey what
does it tell me essentially this um this
transfer knowledge that we're having
from what is been generated and is it
actually applicable can I do something
with it I think that bridge is still
missing but the first step that's need
before understanding what's going on
yeah that's amazing right that
works quite well I would
say oh thanks um then now looking
forward you you already mentioned that
there's like there's like a gap until
until LMS like until you can fully trust
LMS or if you want if you want to add
this this maybe the first question if
you want to fully trust alms um but but
more generally speaking um
what like yob to you the question um
what what further advancements do you
see to undergo for example to get rid of
this the symptom that you would like
only um consult lmms and then make yeah
make your own decision um to for example
being LMS being um able to decide for
themselves and and act on
this difficult question I think um that
we're quite far are still from a place
where llms do everything and take over
all the decision processes and I think
you have to differentiate a little bit
what's the difficulty of a certain task
difficulty of a certain decision right
for simple tasks that don't require 100%
accuracy all the time we're already
there they can already take over quite a
lot in my opinion for example like chbd
plugins insta card just giv me like a a
recipe and then order it on inst card
right right yeah yeah simple
interactions right um there's also maybe
not that much of the line we can already
do that I would say um to get to a point
where llms do much more complex
decisions uh or take them over
completely I don't think we're going to
get there anywhere soon because also
traditional machine learning doesn't
most of the time do that I would say
right I think in some cases very simple
case it in cases it does and we need to
scale up a lot of very simple decisions
yes I see it in a similar sense maybe
less quantitatively based but more text
based decision that are that are being
taken um I think especially in in the
interaction space right really having
this as a sort of first interaction
layer with a user with a customer
internal user external user I think
that's where LMS will take over a lot of
decisions very independently but really
making pivotal business decisions for
example um I don't think we're going to
get there yet I think it's really
similar to machine learning as it does
already taking over over the 80% of the
manual work and then still leaving the
20% and the final decision- making to
the to the human in many cases yeah well
thanks so so so you rather say okay if
your question if you really want it want
them to to make decisions um rather
being like the first point of contact um
yeah exactly exactly yeah um okay so
really interesting because I recently um
watched the talk from the Nvidia CEO and
um he envisions a future or predicts a
future where LMS will be um the first
point of contact for every interaction
with the system um Hannah would you go
uh with his uh his prediction of the
future and maybe also um what do we need
to undergo in terms of uix um until we
reach maybe such a
point well I love this question it's a
very steep hypothesis uh
so I think I have a slightly different
take there um to be honest I don't think
language is always the most efficient
way to communicate with a system if I'm
very honest I think sometimes a simple
drop down and hit and go actually does
the job quite well so I I think we
should keep that in mind like what what
is really what is the good experience
for the user to interact with a system
if there if you have for example an FAQ
um space with only five questions adding
a chap but on top can make it even like
actually quite confusing because you
would have to kind of guess what this
chatbot can help you with instead of
just like browsing quickly having a
glance at these five
questions but in general I I do agree
what's an very interesting trend is to
see language being more and more an
interface between um humans and machines
even between machine and machines in the
sense that we see this trend of
multimodal models involving where you
have models trained on C different
modalities like for example image and
text or so video and text so you can now
ask questions about an image for example
and get the output read to you for
example by an AI voice so there's so
many ways how language kind of becomes
this interface and I think this is an
incredibly interesting Trend so yes I do
think that we will see a further rise of
language based uh interactions but I'm
not quite convinced that prompting and
like iterating on prompts to get the
right answer is necessarily the solution
for for every
system think you just described clippy
basically um so I generally do not think
language is anywhere close to being the
most efficient way to to to with any
sort of system I think and that's
essentially also what I attribute to the
success of chat2 is the the tendency of
humans to feel much more comfortable
with anthropomorphic systems like chat
talks and reads like a human being and
that's what makes it so impressive
whereas a complicated I mean we all Tech
space here CLI or whatever interface you
might have is then sort of scary and
spooky
but I think most of us in this room
might actually prefer a CLI because
that's the most efficient way to maybe
communicate with whatever is that you
want to do do I think text is efficient
absolutely not it takes a long time to
write takes a long to read a drop down
is two clicks so I hope that we do not
get into this direction because I think
we go in circular we can see that uh the
understanding of systems is increasing
you know as I said my dad is quite well
as iPhone these days you know um so we
don't need these systems from like
Clippy which were designed to in the
'90s to make people who do not
understand what you know the trash can
means on your desktop to explain it to
them but but one point where why I
partly disagree let's say is that sure I
agree with an educ with your point for
an educated bubble of very tech savvy
people but now that's not what Humanity
looks like right I think the really
average and the majority of folks that
used to technology use most of the
products they are rather uncen towards
technology right for them is technology
is something that they use as a user but
they don't think about what's the most
efficient way in communicating with it
and for them natural language is the
most natural way for sure and they don't
even think about whether it's the most
efficient or not for them it's the first
way that someone would communicate so in
that light I would say surely it's a
bold statement to say it's every single
time the first um layer of communication
but I would say we're going to hit I
know if you look at all the system inter
system interaction probably over all the
users probably 90% will be hitting a
natural language layer first and then um
access functionality and I mean a good
example of that is um who would have
thought before search engines came out
that some sort of keyword search uh
weird query is the way of interact with
online content right is this the most
efficient way to interact with it
probably not right but it's the way that
really that everyone can use right that
everyone used to be able to use and
similarly I think the way of querying
documents quering knowledge is going to
change quite a bit more towards the
semantic way and more use more towards
using natural language but similar to
search I think we're going to have
similar manner that search bars work now
going have more agents or search bars
that act as agents to interact as a
first layer that's for
sure maybe to just add one thought
because I recently read an interesting
Comon from the niss Norman Foundation
which you might know it's like a famous
ux research organization so one thing
they said about prompting is that you
also have to keep in mind that it
requires a pretty high degree of
literacy to to frame your thoughts in a
way that you get the answer you want at
least at the moment because you yeah we
all know this experience of prompting
and then not exactly getting what we
wanted and then refining the prompts so
so I think one interesting aspect there
is also to make to make sure this is
very inclusive if language becomes more
and more an interface so um thinking
about also the developments in research
towards integrating like more longtail
languages for example um allowing
interactions not only in English but
like in a diverse set of um languages
it's just one strand of research of
things we are seeing that still need to
happen in the space of NLP next to
getting rid of hallucinations or aspects
like that but but I think language and
extending to to these more long TR
languages um is very
important thought there um completely
agree that obviously prompting in the
back right when you're hitting the llms
is complex and need to follow a certain
structure but on the other hand I'm not
expecting from my user to write these
prompts right if I'm building an
building an application and I'm
integrating LMS then then it's on me to
make sure the user can use whatever
natural language they um used to and
then I need to write the prompt that
talks to the to the llm directly and
make sure that it includes um or makes
the llm understand whatever query the
user word in the natural language so I
don't think that or literacy in terms of
languages for sure obviously needs to
inclusiv needs to beur but I actually
think that lmms have the huge advantage
that different types of language
literacy in a sense can be considered
someone with very complex language can
write a switch query and it can be
similarly understood to someone with
very easy language because the llm can
make can actually understand what they
want so I'm not so sure about it though
I think that's I think that's what we
disagree U because I think language is
and I'm not just talking about this
bubble that we in it's just not the most
efficient way to transort information um
I can't recall into how many arguments
I've gotten on WhatsApp because a
message I've written was receiv in a
wrong way and if any sentence you say
and you put the emphasis on something on
a different place it has a completely
different meaning I told you that's a
nice shirt you might be offended whereas
I might actually as a genuine compliment
because I also like wuang you know um
and and I think this is then the issue
that you have when you then interact
with these systems where if you say hey
I want um last quarter's
Revenue what quot are we talking is it
the financial year is it being you know
the calendar year and and these context
is what then need to figure out from
person Business site and then also say
if you talk about the 90% of
consumers what does it mean to them we
talked about um it being very accessible
and I think this sort of reminds me of
um when iOS launched they had this this
heavily geomorphic design so your
contacts were like like a roller de
because we had to sort of accustom
people that swiping means you go through
something you had to pinch to zoom
because well I don't know what this
occurs naturally but we had all of these
metaphors for very familiar objects and
we've moved away from this because now
we have more phones and people on this
planet and we sort of understand this is
the Gen the overall gesture to zoom into
something and this means up and down and
I think with more technical literacy
that we have as as a human race we're
also going to move away from these
abstractions that might make things
actually a bit more difficult my opinion
because prompting and getting the right
thing is actually not as straightforward
as it likely to
be abolutely actually it's funny that
you took that you um chose the Q3
Revenue last quue example because that's
exactly the one of the examples I had in
my talk earlier right but I believe that
with a good semantic understand layer
that understands semantic meaning you'll
be able to cover that with an L because
an LM can either ask you question ask
you back hey there are multiple quarters
right which one do you mean or quarter X
quarter y quarter z um and that's
exactly the major advantage in my
opinion because you can use the natural
language and it can make these
connections put them into context maybe
see what's available in my database in
my knowledge base and um draw
conclusions across essentially so yeah
completely see the challenge 100% um but
that that's really on the prompting
don't give the answer directly but um
consider all the options or however
right okay I I have the feeling that the
conversation could go could go on for
like U plenty more minutes I have a lot
of questions as well but I want to give
you as the audience the chance to ask
questions if there are any um to our
three
panelists otherwise as I said I have I
have plenty more so um but we are happy
to assist you with a microphone and
then you can ask
him if not uh then I will ask my
questions um so we now talked about like
the basic interaction with the system
but now coming back to M um what do you
think will the the role be of llms in
this whole uh sphere of interaction with
the with the system where will the the
role in the future be um will it be for
example for every textual interaction
will it be an llm or uh where do you see
it I think the main benefit of llms at
the moment are are the multimodality
right they can work with texts but they
can also maybe work with tables you
canook up to your dat datase they can
work images and um in a in a in a
scenario maybe 10 20 years from now we
could speak that would be is that this
sits basically a part of infrastructure
it's part of your office suite as part
of all the tables that you have and from
that generates your overall knowledge
base and having essentially your your
your your Google llm your marantic llm
your llm which essentially is then your
go-to person for questions you have and
okay listen where are these slides that
are built for four years ago and they're
oh here they go oh thank you um this is
what I see essentially this main core
productivity benefit that an llm can
offer on a productive scale as part of
any organization really I think that's a
bit about the accessibility here this
can be part of any literally any company
it does matter how boring or fashioned
you are this can be embedded anywhere
right now I think we're not there yet
absolutely not because LS are then
always constricted to some sort of
microcosm that we want to explore maybe
that's your own your database for your
SES Maybe it's only you know some
textual data as part of your HR
department whatever um but I think this
is missing the Integrity of integration
across all departments which is where
going to get the main benefit
overall and I think textual information
is the start of it but then really the
underlying value it's always thinking a
little bit what are the patterns that
where again um technology strength of
the technology come together with the
business value right what the common
patterns you can kind of reuse use use
in every single company and I think this
um building a semantic understanding for
whatever data you're working with
whether it's uh structured data whether
it's time series data where whether it's
textual data whether it's images right
connecting all of that in a semantic
knowledge base that is huge it's huge so
so so sort of sort of a an evolution of
of search or organization of information
right just like a super quick addition I
think what I'm also really looking
forward to seeing is more use cases
where you you have a true interaction
which is kind of both sided so it's not
just us learning how to better prompt to
get a better result but it's also the
model kind of gradually learning as we
go along so this is something we are
seeing as a trend not only in language
but basically overall in machine
learning and I think the coolest systems
and it's something I'm super excited
about in general is whenever you we're
at a stage where you provide some input
and the model maybe gives a suggestion
and then you give some sort of feedback
as a human and the model learns from the
experts gradually as you advance and
then step by step you can automate
further essentially and I think that
would be great to to see that also um
more more with llms in general um but in
general I also absolutely agree with
what you've said already thanks we're
going to take this
question thanks for the discussion was
really cool and I have
question basic like actually it's two
questions but I try to compile it in one
um at which level of like let's say
success rate do you think will llms
actually be implemented is that like an
do you think that's like an absolute
like it shouldn't be like 99% it will
give the correct answer and then 1% it's
going to give you the wrong answer for
like something where a customer asks you
like I let's say at the um as a customer
service right like and you ask a
question how do I do exploit that at
which point do you think companies will
actually
feel safe with llms and Which percentage
and how do you ensure that they actually
trust them because right now I think
there's still like this ah yeah but
maybe what about that 1% when they like
go off and then they're like how do I I
don't know update my iPhone and then it
tells you just throw it into to the
microwave or something I don't know um I
know it's it will be more reasonable
than that but like how do you see that
um and how do you think or what do you
think it's another question how do you
think like the whole topic or like the
whole argument of l&m's just being
stochastic parrots um what do you think
about that do you think they are
stochastic parrots only or do you think
they have like an inner World model that
they actually can somewhat reason we
already ran out of time but Hannah I
would say um you will give give the
answer and then um you're you're always
welcome to then discuss afterwards but
Hannah here you go okay I will I will
hurry so that you also get a chance to
comment if you like so stochastic parrot
yes definitely I mean this is partially
what they are built for that's great uh
they they they give um the most
probabilistic answer which is very very
powerful but also comes with these
limits of potentially giving wrong
answers so I think in general you can
think about it in two aess one is how
good is the model compared to how good a
human could do it and the other is um
kind of how sensitive is the use case
and whenever um the model is better than
human and the case is not that sensitive
or for full automation if the model is
better than human or close to human but
the case is somewhat sensitive you
probably already reached the stage where
you want to have a human in the loop and
then we can have different stages of or
different ways of having the human in
the loop but you probably always want
that and I think
if you go for an approach of that it
takes out a lot of risk while still
utilizing the benefits and this is
something I would always recommend to
clients I think it's not reasonable to
to automate fully if it's a very risky
case or a human could do the job
better I see someone and yob nodding so
we're gonna take this thanks a lot Hanah
um so thanks a lot for the uh L lovely
discussion and like I said I I have the
feeling that we could go on for some
more minutes um one that I want to
mention definitely is um the park on
tomorrow um we we are at C and someone
is giving a talk there as well on the
interesting topic why causality is the
ne next big step in AI uh it is in
Cinema 4 at 2:30 so don't miss it I will
be definitely there um and uh this
leaves me with nothing less than uh
expressing my gratitude to you three uh
for sharing your knowledge to project a
team and Google team and also for your
questions and your
thanks lot
you
浏览更多相关视频
'AI Superpowers': A Conversation With Kai-Fu Lee
アートがよくわからないあなたへ。チームラボ代表が解説するアートの歴史と変化【猪子寿之×堀江貴文】
Full interview: "Godfather of artificial intelligence" talks impact and potential of AI
フルサイズ VS APS-Cの議論、いい加減やめませんか?
OpenAI CEO Sam Altman and CTO Mira Murati on the Future of AI and ChatGPT | WSJ Tech Live 2023
【2029年に人生は3倍になる】科学者玉城絵美氏の未来予想図/日本は人間拡張研究をリードできる/農業と観光が変わる/理工系の学部に女性が少ない理由【玉城絵美×茂木健一郎 後編】
5.0 / 5 (0 votes)