GPT-4o, AI overviews and our multimodal future

Summary

El episodio de 'Mixture of Experts', conducido por Tim Wong, reúne a expertos en AI para debatir los acontecimientos más importantes de la semana. Se discuten dos demostraciones destacadas de Google y OpenAI sobre modelos multimodales que permiten a los usuarios obtener respuestas en tiempo real mediante la cámara de su teléfono. Se exploran las implicaciones de estas tecnologías en el ámbito empresarial y el consumo, así como la importancia de la latencia y los costos en el desarrollo de agentes de IA. Además, se analiza el anuncio de Google sobre los resúmenes de AI en los resultados de búsqueda y su impacto en el mercado y la economía web, planteando preguntas sobre el futuro de la búsqueda y la web como plataforma dinámica y saludable.

Takeaways

• 🤖 La importancia de la multimodalidad: Las compañías están enfocadas en modelos de IA que puedan tomar entradas de video y hacer sentido de ellas, lo que puede cambiar drásticamente el uso de la IA en el futuro.
• 🚀 El impacto de la latencia y los costos: La reducción de los costos y la mejora en la velocidad de los modelos de IA tienen el potencial de afectar enormemente las aplicaciones descendentes de la IA.
• 🔍 La introducción de resúmenes de AI en Google Search: Google anunció que comenzará a mostrar resúmenes generados por AI en los resultados de búsqueda, lo que representa un cambio significativo en la experiencia del usuario.
• 🛠️ La aplicación de IA en flujos de trabajo empresariales: La IA está comenzando a ser integrada en procesos empresariales, lo que puede mejorar la eficiencia y la toma de decisiones.
• 👓 La visión del futuro con dispositivos multimodales: Se discute cómo los dispositivos como los Google Glasses pueden ser utilizados en el futuro para una interacción más fluida y eficiente con la IA.
• 📱 La competencia entre la experiencia de usuario de los asistentes de IA y las aplicaciones tradicionales: Se destaca cómo la interacción con la IA puede ser más satisfactoria que la navegación a través de múltiples botones en una aplicación.
• 💬 La mejora en la latencia para fomentar conversaciones más humanas: Se resalta la importancia de la baja latencia para que las interacciones con la IA sean más naturales y fluidas.
• 🔑 La personalización en los resultados de búsqueda con AI: Se sugiere que el uso de AI puede llevar a una personalización más efectiva en los resultados de búsqueda, mejorando la experiencia del usuario.
• 🌐 La consideración de la transformación de la web con la introducción de resúmenes de AI: Se reflexiona sobre cómo la integración de resúmenes de AI en los resultados de búsqueda de Google puede cambiar la forma en que interactuamos con la web.
• 📈 La transformación potencial de la búsqueda en un bien de consumo común: Se predice que la búsqueda puede volverse un servicio más genérico y accesible a través de asistentes de IA, en lugar de la búsqueda tradicional en navegadores.

Q & A

¿Quién es el anfitrión del programa Mixture of Experts?

El anfitrión del programa Mixture of Experts es Tim Wong.

¿Qué tipo de invitados participan en el programa Mixture of Experts?

El programa reúne un equipo de investigadores, expertos en productos, ingenieros y más.

¿Qué compañías fueron discutidas en el episodio en relación con sus anuncios recientes?

Las compañías discutidas en el episodio son OpenAI y Google, en relación con sus recientes anuncios en el campo de la inteligencia artificial.

¿Cuáles son los tres temas principales que se discuten en el episodio en relación con los anuncios de Google y OpenAI?

Los tres temas principales son la multimodalidad, la latencia y los costos, y el anuncio de Google sobre los resúmenes de AI en los resultados de búsqueda.

¿Qué es la multimodalidad y cómo se relaciona con los modelos de IA?

La multimodalidad se refiere a la capacidad de los modelos de IA para tomar diferentes tipos de entrada, como video, y ser capaces de entender y procesar esa información de manera efectiva.

¿Por qué la reducción de la latencia y los costos en los modelos de IA es significativa para el uso posterior de la IA?

La reducción de la latencia y los costos puede tener un gran impacto en el uso posterior de la IA, ya que permite que los modelos sean más accesibles y se puedan integrar en aplicaciones más rápidas y eficientes.

¿Qué cambios significativos anunció Google en relación con los resúmenes de AI en los resultados de búsqueda?

Google anunció que los usuarios de Google Search comenzarían a ver resúmenes de AI en la parte superior de los resultados de búsqueda, lo que representa un cambio significativo en la experiencia del usuario.

¿Qué es Gemini y cómo se relaciona con la demostración de Google?

Gemini es un modelo de IA presentado por Google en su demostración, que muestra la capacidad de la IA para interactuar en tiempo real con la cámara de un dispositivo móvil.

¿Cómo podría la tecnología multimodal impactar en los flujos de trabajo empresariales según Shobhit Varshney?

Según Shobhit Varshney, la tecnología multimodal podría impactar significativamente en los flujos de trabajo empresariales al convertir los teléfonos en extensiones de los sentidos, facilitando la automatización de tareas y mejorando la eficiencia en procesos como la auditoría de planogramas.

¿Qué es un planograma y cómo se relaciona con la aplicación empresarial de la IA según lo discutido en el episodio?

Un planograma es una representación visual de cómo se deben colocar los productos en los estantes de una tienda. La IA puede ayudar a automatizar el proceso de auditoría de planogramas, comparando imágenes de los estantes con las posiciones correctas de los productos.

¿Qué cambios en la experiencia del usuario se esperan con la implementación de resúmenes de AI en los resultados de búsqueda de Google?

Se esperan cambios significativos en la experiencia del usuario, donde en lugar de tener que navegar a través de múltiples resultados de búsqueda, los usuarios podrán recibir respuestas directas y personalizadas a sus consultas en los resultados de búsqueda de Google.

Outlines

00:00

🤖 Introducción a Mixture of Experts

El episodio inicia con la presentación del programa 'Mixture of Experts' por su anfitrión, Tim Wong, quien menciona que cada semana reúne a un equipo de expertos en IA para debatir los principales acontecimientos de la semana. En este caso, se centrará en la rivalidad entre OpenAI y Google, discutiendo los anuncios recientes y su impacto en la industria. El panel incluye a Shobhit Varshney, un consultor senior en AI, Chris Hay, un distinguido ingeniero y CTO de Transformación de Clientes, y Bryan Casey, Director de Marketing Digital, quien se unirán para analizar los temas clave de la semana.

05:04

🔍 Multimodalidad y su impacto en el futuro

En este segmento, se discute el tema de la multimodalidad, con enfasis en cómo ambos gigantes tecnológicos, Google y OpenAI, están trabajando en modelos que puedan manejar múltiples tipos de entrada, como videos e imágenes, y producir salidas que combinen texto, audio y otros modales. Se menciona que esta tecnología no solo es impresionante sino que también puede desbloquear nuevos productos y cambiar significativamente los flujos de trabajo en la vida diaria y en los negocios, como la automatización de procesos en empresas, mejorando la eficiencia y la interacción con el usuario final.

10:08

🛠️ Aplicaciones empresariales de la tecnología multimodal

Se exploran las aplicaciones empresariales de la tecnología multimodal, destacando cómo la capacidad de los modelos de IA para interpretar y responder a datos de video en tiempo real puede revolucionar tareas específicas y flujos de trabajo en empresas. Se discute cómo en el pasado era necesario codificar procesos y documentarlos detalladamente para la automatización, pero con los avances en IA, es posible una interacción más dinámica y menos estructurada, permitiendo la interrupción y adaptación de tareas en curso. También se menciona el uso de esta tecnología en escenarios de ventas al por mayor y cómo puede facilitar la toma de decisiones y la auditoría de productos en tiendas.

15:09

🕒 Importancia de la latencia y los costos en el desarrollo de AI

En este apartado, se aborda el tema de la latencia y los costos asociados con los modelos de IA. Se destaca la importancia de tener modelos que puedan responder rápidamente para permitir interacciones fluidas y conversaciones naturales con los usuarios. Se menciona que los modelos más grandes, aunque capaces de razonamiento más profundo, tienen una latencia mayor, mientras que los modelos más pequeños y rápidos pueden no ser tan efectivos en la razón. Se debate cómo equilibrar entre estos factores para lograr una implementación eficaz de la tecnología en aplicaciones prácticas.

20:11

📈 Estrategias empresariales para adoptar modelos de IA

Se discuten las estrategias que las empresas pueden emplear para integrar modelos de IA en sus operaciones. Se sugiere que en lugar de reemplazar completamente los flujos de trabajo existentes, es posible mejorar ciertos aspectos específicos utilizando modelos de IA más pequeños y especializados. También se destaca la importancia de evaluar aspectos como la latencia, el costo y la seguridad al seleccionar el modelo de IA adecuado para una tarea determinada. Se enfatiza la necesidad de una comprensión clara de los procesos y los resultados esperados para una implementación exitosa de la tecnología.

25:14

🎤 Perspectiva de marketing sobre la interacción con IA

Desde una perspectiva de marketing, se analiza cómo la tecnología de IA está cambiando la experiencia del usuario y las expectativas en términos de interacción. Se menciona la importancia de la latencia y cómo una respuesta rápida de los sistemas de IA puede mejorar significativamente la experiencia del usuario, haciendo que la interacción con un agente virtual sea más aceptable y efectiva. Se sugiere que la capacidad de conversar con un AI y recibir respuestas inmediatas puede ser un factor clave en la adopción de esta tecnología en el mercado.

30:17

🌐 La evolución de la búsqueda con la introducción de resúmenes generados por AI

Se debate el anuncio de Google sobre la implementación de resúmenes generados por AI en los resultados de búsqueda. Se plantea la idea de que esta medida puede cambiar la economía del web y la forma en que los usuarios interactúan con la información en línea. Se reflexiona sobre el impacto que tendrá en el SEO y en la forma en que los usuarios encontrarán y consumirán contenido en el futuro, sugiriendo que los resúmenes de AI podrían ofrecer una experiencia más enriquecedora y personalizada que la búsqueda tradicional.

35:17

📚 Perspectivas finales sobre la IA y la búsqueda

En el último párrafo, los panelistas comparten sus perspectivas finales sobre cómo la IA y los asistentes virtuales van a cambiar la forma en que interactuamos con la búsqueda y la información en general. Se sugiere que la búsqueda se convertirá en un bien más común y que la interacción con los asistentes será la nueva forma de obtener información. Se destaca la importancia de la personalización y la capacidad de los asistentes para comprender y satisfacer las necesidades de los usuarios de manera más eficiente que los motores de búsqueda tradicionales.

Keywords

💡Mixture of Experts

Mixture of Experts es el nombre del programa en el que se discuten temas relacionados con la inteligencia artificial. Es un espacio donde expertos de diversos campos se reúnen para debatir y analizar las últimas noticias y tendencias en AI. En el guion, se menciona como el medio para abordar temas como los anuncios de Google y OpenAI y su impacto en la industria.

💡OpenAI

OpenAI es una organización de investigación y desarrollo de inteligencia artificial que se menciona en el guion como una de las compañías en el 'showdown' de la semana. Se discuten sus anuncios y cómo están compitiendo o colaborando con Google en el campo de la IA, destacando su modelo 4.0 y su capacidad para procesar múltiples modalidades de entrada.

💡Google

Google es una de las compañías líderes en tecnología y se menciona en el guion como la otra mitad del 'showdown' con OpenAI. Se discuten sus anuncios recientes y cómo su modelo Gemini compara con los modelos de OpenAI, así como su enfoque en la reducción de latencia y costes en sus modelos de IA.

💡multimodality

La multimodalidad se refiere a la capacidad de los modelos de IA para manejar y comprender múltiples tipos de entrada, como video, audio, texto, etc. En el guion, se discute cómo tanto Google como OpenAI están enfocados en mejorar la multimodalidad en sus modelos para permitir una interacción más rica y dinámica con los usuarios.

💡latency

La latencia se refiere al tiempo que tarda un sistema en responder a una entrada o solicitud. En el guion, se destaca la importancia de reducir la latencia en los modelos de IA para permitir interacciones más fluidas y naturales, lo que es crucial para la adopción en escenarios de usuario final y en el entorno empresarial.

💡costs

Los costos asociados con la computación y el uso de modelos de IA son un tema importante en el guion. Se discute cómo las compañías están trabajando en modelos más pequeños y eficientes para reducir los costos por cada token o interacción, lo que puede tener un impacto significativo en la implementación y adopción de la tecnología.

💡AI summaries

Los resúmenes de IA son una característica anunciada por Google donde los usuarios de Google Search comenzarán a ver resúmenes generados por IA en los resultados de búsqueda. En el guion, se cuestiona si esto es positivo y se explora el impacto que podría tener en la experiencia del usuario y en la economía de la web.

💡enterprise

El término 'enterprise' se utiliza en el guion para referirse a las aplicaciones de la IA en el sector empresarial. Se discuten ejemplos de cómo las mejoras en la tecnología de IA pueden optimizar procesos empresariales, como la automatización de flujos de trabajo y la mejora de la precisión en tareas específicas.

💡consumer

El contexto 'consumer' se contrasta con 'enterprise' en el guion al discutir los usos y aplicaciones de la IA. Se menciona cómo la tecnología multimodal puede afectar tanto a los consumidores finales como a las empresas, y se cuestiona dónde podría haber un mayor impacto o adopción inicial.

💡search engine

Los motores de búsqueda son una temática central en el guion, especialmente en relación con el anuncio de Google de implementar resúmenes de IA. Se reflexiona sobre cómo este cambio podría afectar la forma en que los usuarios interactúan con la web y si esto podría desplazar la necesidad de un motor de búsqueda tradicional.

Highlights

Mixture of Experts podcast discusses the latest AI news and developments.

OpenAI and Google's recent announcements indicate a focus on multimodal AI models that can process video inputs.

Debate on the potential of AI to reduce latency and costs, making AI more accessible and faster.

Google's introduction of AI summaries in search results represents a significant shift in user interaction with the web.

Shobhit Varshney emphasizes the impact of AI advancements on enterprise workflows, particularly in knowledge transfer.

Chris Hay discusses the potential of multimodal AI in transforming consumer and enterprise interactions.

Bryan Casey brings a marketer's perspective on the importance of AI's ability to provide real-time responses.

The conversation explores the balance between model size, reasoning ability, and response latency.

Enterprises are focusing on applying AI at the sub-task level to improve specific steps in a workflow.

Discussion on the potential of AI to automate complex decision-making processes, such as planning events or making purchases.

Brian Casey argues that Google's integration of AI overviews is crucial for maintaining the web's dynamism and health.

Shobhit Varshney predicts a future where AI can hyper-personalize search results based on individual user data.

Chris Hay envisions a future where search becomes a commodity, and AI assistants handle user queries.

The panelists agree that AI's impact on search and user experience will be significant and disruptive.

Final thoughts on the importance of feedback mechanisms in AI to improve personalization and user experience.

Transcripts

Hello and welcome to Mixture of Experts.

I'm your host, Tim Wong.

Each week, Mixture of Experts brings together a world class team of

researchers, product experts, engineers, uh, and more to debate and distill down

the biggest news of the week in AI.

Today on the show, the OpenAI and Google showdown of the week.

Who's up, who's down, who's cool, who's cringe?

What matters, and what was just hype?

We're going to talk about the huge wave of announcements coming out of

both companies this week, and what it means for the industry as a whole.

So, for panelists today on the show, I'm ably supported by an incredible

panel, uh, two veterans who have joined the show before, and a new,

uh, contestant has joined the ring.

Um, so, first off, uh, Shobhit Varshney, he's the Senior Partner Consulting

for AI in US, Canada, and LATAM.

Shobhit, welcome back to the show.

Thanks for having me back, Tim.

Love this.

Yeah, definitely.

Glad to have you here.

Chris Hay, who is a distinguished engineer and the CTO of Customer Transformation.

Chris, welcome back.

Hey, nice to be back.

Yeah, glad to have you back.

And joining us for the first time is Bryan Casey, who is the Director

of Digital Marketing, who has promised a 90 minute monologue.

Uh, on AI and search summaries, which I don't know if we're going to get to,

but we're going to have him have a say.

Bryan, welcome to the show.

We'll have to suffer through showbit and Chris for a little bit, and

then we'll get to the monologue.

But for having me.

The good stuff.

Yeah, exactly.

Um, well, great.

Well, let's just go ahead and jump right into it.

So obviously there were a huge number of announcements this week.

OpenAI came out of the gate with its kind of raft of announcements.

Uh, Google I.

O.

is going on and they did their set of announcements.

And so, really, more things, I think, were debuted, promised, coming

out, than we're going to have the chance to cover on this episode.

But sort of from my point of view, and I think I wanted to use this as a way

of organizing the episode, there were kind of three big themes coming out

of Google and OpenAI this week that we'll sort of take in turn and use

to kind of make sense of everything.

So I think the first thing is multimodality, right?

Both companies are sort of obsessed with their models taking video input and being

able to make sense of it and going from, you know, image to audio, text to audio.

Um, and I want to talk a little bit about that.

Second thing is latency and costs, right?

Everybody touted the fact that their models are going to be cheaper, and

they're going to be way faster, right?

And, you know, I think if you're from the outside, you might say, well,

it's kind of a difference in kind.

Things get faster and cheaper.

But I think what's happening here really potentially might have a huge

impact on downstream uses, uh, of AI.

And so I want to talk a little bit about that dimension and sort of what it means.

Um, and then finally, uh, I've already kind of previewed a little bit.

Um, Google made this big announcement that I think is almost literally

going to be like Many people's very first experience with LLMs in full

production, uh, Google basically announced that going forwards, uh, the U.

S.

market and then globally, uh, those users of Google search will start

seeing AI summaries at the top of each of their sort of search results.

Um, that's a huge change.

We're gonna talk a little bit about what that means and, um, if it's good.

I think is a really good question, uh, so looking forward to diving into it all.

So let's talk a little bit about multi modal first.

So there's two showcase demos from Google and OpenAI, and I think both of them

kind of roughly got at the same thing, which is that in the future you're going

to open up your phone, you're going to turn on your camera, and then you can

wave your camera around, and your AI will basically be responding in real time.

And so, Shobhit, I want to bring you in because you were the one who

kind of flagged this being like, we should really talk about this, because

I think the big question that I'm sort of left with is like, you know,

where do we think this is all going?

Right?

It's a really cool feature, but like, what kind of products do we

think it's really going to unlock?

And maybe we'll start there, but I'm sure, I mean, this topic goes

into all different places, so I'll give you the floor to start.

Tim, Monday and Tuesday were just phenomenal inflection points

for the industry altogether.

It's getting to a point where an AI can make sense of all

these different modalities.

It's an insanely tough problem.

We've been at this for a while and we've not gotten it right.

We spent all this time trying to create pipelines to do each of these speech to

text and understand and then text to back.

It takes a while to get all of the processing done.

The 2024 we were able to do this, what a time to be alive man.

I just feel that we are getting, finally getting to a point where.

Your phone becomes an extension of your eyes, of your listening

in and stuff like that, right?

And that is a, that has a profound impact on some of the

workflows in our daily lives.

Now, within IBM, I focus a lot more on enterprises.

So I'll give you more of an enterprise view of how these

technologies are actually going to make a, make a difference or not.

In both cases, Gemini and OpenAI is 4.

0.

And by the way, in my case, I'm 4.

0.

4.

0 does not stand for Omni.

For me, 4.

0 means, oh my God, it is really, really that good.

So, um, we're getting to a point where there are certain workflows

that we do with enterprises, like you are looking at transferring

knowledge from one person to the other.

And usually you're looking at a screen and you have a bunch of here's

what I did, how I solved for it.

We used to spend a lot of time trying to capture all of that

and what happened in the desktop.

Classic BPO processes, these are billions of dollars of work that happens, right?

Yeah, and I think if I could pause you there, like I'm curious if you can

explain, because again, this is not my world, I'm sure a lot of listeners

aren't, it isn't their world as well.

How did it used to be done?

Right?

Like, so if you're, you're trying to like automate a bunch of these

workflows, is it just people writing scripts for every single task?

Or like, I'm just kind of curious about what it looks like.

Yeah.

So Tim, let's, let's pick a more concrete example.

Uh, say you have outsourcing a particular piece of work and your finance documents

coming in, you're comparing it against other things, you're finding errors,

you're going to go back and send an email, things of that nature, right?

So we used to spend a lot of time documenting the current process.

And then we look at that 7...20...9 step process and say I'm going to call an

API, I'm going to write some scripts, and all kinds of issues used to happen

along the way, unhappy paths and so forth.

So the whole process used to be codified in some level of code, and

then it's deterministic, it does one thing in a particular flow really well.

And you can't interrupt it, you can't just barge in and say, no, no, no, this is not

what I wanted, can you do something else?

So we're now finally getting to a point where that knowledge work, that work

that is to get done in a process, that'll start getting automated significantly with

announcements from both Google and OpenAI.

So far people would solve it as a decision step by step flowchart.

But now we're in a paradigm shift where I can, in the middle of it,

interrupt an action to say, Hey, see what's on my desktop and figure it out.

I've been playing, I've been playing around with, uh, with OpenAI's 4.

0, its ability to go look at a video of a screen and things of that nature.

It's pretty outstanding.

We are coming to a point where the, the speed at which the

inference is happening is so quick.

And now you can physically we can actually bring them into your

workflows early to just take so long.

It was very clunky is very expensive.

So you couldn't really justify adding AI into those workflows.

It'll be you do liver arbitrage or things of that nature

versus trying to automate it.

So these are kind of workflows infusing AI in doing this entire

process into an phenomenal unlock.

One of my clients is a big CBG company.

And as we walk into the aisles, they do things like panograms where you're

looking at a picture of the Um, And these consumer product goods companies

would give you a particular format in which you want to keep different

chips and drinks and so forth.

And each of those labels are turned around or they are in a different place.

You have to audit and say, Am I placing things on the shelf the right way?

Like the consumer product goods model.

That's called planogram.

Realogram, planogram, that's the idea.

So earlier we used to take pictures, a human would go in and note things

and say, yes, I have enough of the bottles in the right order.

Then we started to take pictures and analyzing it.

You start to run into real world issues.

You don't have enough space to back up and take a picture or.

You go to the next island, the lighting is very different and stuff like that.

So AI never quite scaled.

And this is the first time now we're looking at models like Gemini and

others where I can just walk past it and create a video and just feed the whole

five minute video in with this context length of two million plus and stuff.

It can just actually ingest.

Right, just process it all.

On the fly.

It was missing.

Yeah.

That's right.

Yeah.

Those kind of things that were very, very difficult to do for us earlier.

Those are becoming a piece of cake.

um, how do I make sure that the AI, the normal stuff that we are

seeing, is grounded in enterprise?

So it's my data, my planogram style, or my processes, my documents, not

getting knowledge from elsewhere.

So in all the demos, one of the things that I was missing was, how do I make it

go down a particular path that I want?

Right.

If the answer is not quite right, how do I control it?

So I think a lot more around.

How do I bring this to my enterprise clients and deliver value for them?

Those are some of the open questions.

Chris, do you have something similar to that?

Totally, I do want to get into that.

I see Chris coming off mute though, so I don't want to break his role.

I don't know if Chris, you got kind of a view on this, or if you

disagree, you're like, ah, it's actually not that impressive.

Google glasses back, baby.

Yeah.

So I, I think, I think multimodality is a huge thing, as Shobhit

covered it correctly, right?

There's so many use cases.

Yeah.

in the enterprise, but also in consumer based scenarios.

And I think one of the things we really need to think about is we've been

working with LLMs for so long now, which has been great, but the 2D tech

space isn't enough for generative AI.

It's, it's, we want to be able to interact real time.

We want to be able to interact with audio.

Um, you know, and you can take that to things like contact centers where you

want to be able to transcribe that audio.

You want to then have AIs be able to respond back in a human way.

And you want to chat with the assistants.

Like you'd like you saw in the open AI demo.

Uh, you know, you don't want to be sitting there and go, well, you

know, my conversation is going to be as fast as my fingers can type.

You want to be able to say, Hey.

you know, what do you think about this?

What about that?

And you want to imagine new scenarios.

So you want to say, what does this model look like?

What does this image look like?

You know, tell me what this is.

And you want to be able to interact with the world around you.

And to be able to do that, you need multimodal, uh, models.

And, and therefore, Like in the Google demo, where, you know, yeah, she picked

up the glasses again, you know, so I jokingly said Google glasses back,

but, but it really is, it's, if you're going and having a shopping experience,

retail, and you want to be able to look at what the price of a mobile phone

is, for example, you're not going to want to stop getting phone out, type,

type, type, you just want to be able to interact with an assistant there and

then, or seeing your glasses, what the price is, and I give the mobile phone

example for a reason, I Which is, the price that I pay for a mobile phone isn't

the same price as you would pay, right?

Because it's all contract rates.

And if I go and speak, if I want to get the price of how much am I paying

for that phone, it takes an advisor like 20 minutes, because they have to

go look up your contract details, etc.

They have to look up what the phone is, and then they do a deal.

In a world of multi modality, where you've got something like

glasses on, it can recognize the object, it knows who you are.

And then it can go and look up what, uh, what the price of the phone is for you.

And then be able to answer questions that are not generic questions, but

specific about you, your contract.

To you.

Right.

Exactly.

That, that is where multi modality is going to start, start to come in.

I mean, it kind of sounds like, right, yeah, totally.

I mean, Chris, if I have you right, I mean, this is one of the questions I want

to pitch to both you, Shobhit, and you, Chris, on this is, You know, I actually,

my mind goes directly back to Google Glass, like the bar where the guy got

beat up for wearing Google Glass years ago, that was like around the corner from

where I used to live in San Francisco.

And you know, there's just been this dream and obviously all the open

AI demos and Google demos for that matter are all very consumer, right?

You're walking around with your glasses and you're looking around

the world and you know, get prices and that kind of thing.

This has been like a longstanding Silicon Valley dream and it's

been very hard to achieve.

And I guess the one thing I wanted to run by you is like, And the answer

might just be both, or we don't know, is like, if you're more bullish on the

B2B side or on the B2C side, right?

Because I hear what Shobit's saying, and I'm like, oh, okay, I can see

why enterprises really get a huge bonus from this sort of thing.

Um, and, and I guess it's really funny to me, because I think there's one

point of view, which is everybody's talking about the consumer use case,

but the actual near term impact may actually be more on the enterprise side.

But I don't know if you guys buy that, or if you really are, like,

this is the era of Google Glass, you know, it's back, baby, so.

I can start first, Tim.

Um, like I've, we've been working with Apple Vision quite a bit, um,

uh, with an IBM with our clients.

And a lot of those are enterprise use cases in a very controlled environment.

So things that where things break in the consumer world, you don't

have a controlled environment.

You have corner cases that happen a lot, right?

In an enterprise setting, If I'm wearing my Vision Pros for two

hours at a stretch doing, I'm a mechanic, I'm fixing things, right?

That's a place where I need additional input and I can't go look at other things

like pick up my cell phone and work on it.

I'm underneath, I'm fixing something in the middle of it, right?

So those use cases, because the environment is very controlled, I can do

AI with higher accuracy, it's reputable.

I know I can start crossing the answers because I have enough

data coming out from it, right?

So you're not trying to solve every problem.

But I think we'll see a higher uptake of these devices.

By the way, I love the Ray Ban glasses from Meta as well.

Great to do something quick, but when you don't want to switch.

But I think we're moving to a point where enterprises will go deliver these

at scale, the tech starts to get better.

And adoption is going to come over on the B2C side.

But in the consumer goods, we'll have multiple attempts at this.

Like we had with Google Glasses and stuff.

It'll take a few attempts to get better.

On the enterprise side, we will learn and make the models a lot better.

But I think there's an insane amount of value that we're delivering

to our clients with Apple Vision Pro today in enterprise settings.

I think it's going to follow that.

Totally.

Yeah.

And it's actually interesting.

I hadn't really thought about this.

And Chris, I'll let you get in.

It's like, um, basically like the phone is almost not as big of

competition in the enterprise setting.

Right.

Whereas like the example that Chris gave was like literally there you're trying to

be like, is this multimodal device faster than using my phone in that interaction?

Which is like a real competition, but if it's something like a mechanic,

you know, they don't have, they don't, they can't just pull out their phone.

Um, Chris, any final thoughts on this?

And then I want to move us to our next topic.

Yeah.

And I was just going to give another kind of use case scenario.

I, I often think of things like the oil rigs example, example.

So a real sort of enterprise space where you're wandering around and you have to

go and do safety checks on various things.

Most of their time, if you think of the days before the mobile phone or

before the tablet, what they would have to do is go look at the part, do the

inspection, the visual inspection, and then walk back to a PC to go fill that in.

And then these days, you do that with a tablet on the rig, right?

But, but then actually, you need to find the component you're going to look

at, you have to do the defect analysis.

You want to be able to take pictures of that.

You need the geo location of where that part is so that

the next person can find it.

And then you want to be able to see the notes that they had before on this.

And then you've got to fill in the safety form, right?

So they have to fill in a ton of forms.

So there's a whole set of information.

If you just think about AI, just having.

You know, even your phone or glasses pick either to be able to look at

that part, be able to have the notes contextualized in that geospatial

space, be able to fill in that form, be able to do an analysis with AI.

It's, it's got a huge impact on enterprise cases and probably multimodality in that

sense has probably got a bigger impact, I would say, in the enterprise cases

than the consumer spaces even today.

And I, and I think that's something we really need to think about.

The other one is.

And again, I know you wanted this to be quick there, Tim,

is the clue and generative AI is the generative part, right?

So actually I can create images, I can create audio, I could create

music things that don't exist today.

So, and with the text part of something like an LLM, Then I

can create new creative stuff.

I can create DevOps pipelines, Docker files, whatever.

So there comes a part where I want to visualize the thing that I create.

I don't want to be copying and pasting from one system to another, right?

That's not any different from the oil rig scenario.

So as I start to imagine new new business processes, new pipelines,

new, uh, tech processes, I then want to be able to have the real time

visualization of that at the same time, or be able to interact with that.

And that's why multi modality is, is really important, probably

more so in the enterprise space.

Yeah, that's right.

I mean, I think some of the experiments you're seeing with, like,

dynamic visualization generation are just, like, very cool, right?

Uh, cause then you basically have You can say, like, here's how I

want to interact with the data.

The system kind of just generates it, right, on the fly, which I

think is very, very exciting.

All right, so next up, I want to talk about latency and cost.

So this is another big trend, you know, I think it was very interesting that

both companies went out of their way to be like, we've got this offering

and it's way cheaper for everybody, which I think suggests to me that, you

know, these big, huge competitors in AI all recognize That like your, your per

token cost is gonna be this huge bar to getting the technology more distributed.

Um, so certainly one of the ways they sold 4.0 was that it was

cheaper and as good as GPT, right?

And everybody was kind of like, okay, well why do I pay for Pro anymore if

I'm just gonna get this for, for free?

And then Google's bid, of course, was Gemini 1.5 flash, right?

Which is okay, it's gonna be cheaper and faster again.

Um, and I know Chris, you threw this.

Uh, sort of topic out, so I'll kind of let you have the first say, but I think

the main question I'm left with is, like, what are the downstream impacts of

this, right, for someone who's not really paying attention to AI very closely,

like, is this just a matter of, like, it's getting cheaper, or do you think,

like, these are actually, these economics are kind of changing how the technology

is actually going to be rolled out?

I think latency and smaller models and tokens are probably one of the most

interesting challenges we have today.

So if you think about like the GPT 4 and everybody was

talking like, oh, that's a 1.

8 trillion model or whatever it is, that's great.

But the problem with these large models is every layer that you have in the neural

network is adding time to get a response back, and not only time, but cost.

So if you look at the demo that OpenAI did, for example, what was really cool

about that demo was the fact that when you were speaking to the assistant, it was

answering pretty much instantly, right?

And that is the real important part.

And when we look at previous demos, what you would have to do if you were

having a voice interaction is you'd be stitching together kind of three

different pipelines you need to do Uh, speech to text, then you're going to run

that through the model, and then you're going to do text to speech back the

way, so you're getting latency, latency, latency, before you get a response, and

that timing that it would take, because it's not in the sort of 300 millisecond

mark, it was too long for a human being to be able to interact, so you got this

massive pause, so actually, the, latency and the kind of tokens per second becomes

the most important thing if you want to be able to interact with models quickly

and be able to have those conversations.

And that's sort of why also multimodality is really important, because if I

can do this in one model as well, then it means that I'm not sort

of jumping pipelines all the time.

So the smaller you can make the model, the faster it's going to be.

Now, if you look at the GPT 4.

on the model.

I don't know if you've played with just the text mode.

It is lightning fast when it comes.

Very fast now.

Yeah, it is.

It's noticeably so like, it's just like, it feels like every time I'm in,

there's like these improvements, right?

Yeah.

And this is what you're doing.

You're sort of trading off reasoning versus, uh, speed of the model, right?

And, and as we move into kind of agentic platforms, as we

move into the multimodality.

You need that latency to be super, super sharp, because you're not

going to be waiting all the time.

So, there is going to be scenarios where you want to move back to

a bigger model, that is fine.

Um, but you're going to be paying the cost.

And that cost is going to be the cost, uh, the price of the tokens in the first

place, but also the speed of the response.

And I think this is the push and pull that model creators

are going to be playing against.

All of the time and, and, and therefore if you can a similar result from a

smaller model and you can get a similar result from a faster model and a cheaper

model, then you're going to go for that.

But in those cases where it's not, then you may need to go to the

larger model to kind of reason.

So this, this is really

Totally.

Yeah.

I think there's a bunch of things to say there.

I mean, I think one thing that you've pointed out clearly is that like

this makes conversation possible.

Right, like that you and I can have a conversation in part

because I have low latency is kind of the way to think about it.

And like now that we're reaching kind of human like parity on latency,

you know, finally these models can kind of converse in a certain way.

The other one is actually, I really thought about that there is kind of

this almost like thinking fast and slow thing where basically like the

models can be faster, but they're just not as good at reasoning.

Um, and then there's kind of this like deep thinking mode, which

actually is like slower in some ways.

So Tim, uh, the way we are helping.

Enterprise clients, again, have that kind of focus in life.

There is a split.

There's a, there's a, there are two ways of looking at applying

Gen AI in the industry right now.

One is at the use case level.

You're looking at the whole workflow end to end, seven different steps.

The other is going and looking at it at a sub task level.

Right, so I'll just pick an example and walk you through it.

So say I have an invoice that comes in and I'm taking an application,

I'm pulling something out of it, I'm making sure that that's as per the

contract, I'm going to send you an email saying your invoice is paid, right?

So some sort of a flow like that, right?

So say it is seven steps, just very simplified, right?

I'm going to pull things from the back end systems using APIs.

Step number three, I'm going to go A fraud detection model that has

been working great for three years.

Step number four, I'm extracting things from a paper.

Right, an invoice that came in.

That extraction I used to be doing with OCR.

85 percent accuracy, humans will do the overflow of it.

At that point we're taking a pause and saying, we have reason to believe

that LLMs today can look at an image and extract this with higher accuracy.

Yeah.

Say we get up to 94%.

So that's nine points, uh, higher accuracy of pulling things out.

So we pause at that point and say, let's create a set of constraints for step

number four to find the right athletes.

And the constraint could be, what's the latency?

Like we just spoke, how quickly I need the result, or can this take

30 seconds and I'll be okay with it?

Yeah.

Second could be around cost.

If I'm doing this a thousand times, I have a cost envelope to

work with versus a human doing it.

If I'm doing it a million times, I can invest a little bit more if I

can get accuracy out of it, right?

So the ROI becomes important.

Then you're looking at security constraints around, does this data

have any identifiable PHI data, PII I have to bring things closer.

Or is this something that is military grade secrets and

has to be on prem, right?

So you have certain constraints around that.

So you come up with a list of five, six constraints, and then

that lets you decide whether.

What kind of an LLM will actually check off all these different

constraints, and then you start comparing and bringing it in.

So the spirit that we're seeing in the market is one way with LLM agents and

with these multi modal models, they're trying to accomplish the entire flow

work for end to end, like you saw with Google's returning the shoes.

It's taking an image of it, it's going and looking at your Gmail to find the receipt,

starting the return, giving you a QR code with the whole return process done.

So just figure it out how to go create the entire end to end workflow.

But where the enterprises are still focused is more on the sub task level.

That point we are saying, This step number four is worth switching and I have enough

evals before and after, I have enough metrics to understand, and I can control

that, I can audit that much better.

The thing that from an enterprise perspective, these end to end multimodal

models, it'll be difficult for us to explain to SEC, for example, why

we rejected somebody's benefits on a credit card, things of that nature.

So I think in the, in the enterprise world, we're going to go down the

path of, Let me define the process.

I'm going to pick small models to Chris's point to do that particular piece better.

And then eventually start moving over to, and now let me make sure that those, that

framework evals and all that stuff can be applied to end to end multimodal models.

I guess I do want to maybe bring in Brian here.

You like release the Brian on this conversation.

Um, cause I, I'm curious about like, kind of like the marketer's view on all this.

Right.

Cause I think there's one point of view, which is yes, yes, Chris, show a

bit like this is all nerd stuff, right?

Like I, you know, it's like latency and cost and speed and whatever.

The big thing is that you can actually talk to these AIs.

Right.

And I guess I'm kind of curious from your point of view about like, I mean,

one really big thing that came out of like the open AI announcements was.

Yeah.

We're going to use this latency thing largely to kind of create this

feature that just feels a lot more human and lifelike, um, than, you

know, typing and chatting with an AI.

And I guess I'm kind of curious about, like, you know, what you

think about that move, right?

Like, is that ultimately, like, Like going to help the adoption of AI?

Is it just kind of like a weird sci fi thing that OpenAI wants to do?

And also, I mean, I think if you've got any thoughts on, you know, how

it impacts the enterprise as well, which is like, do companies suddenly

say, Oh, I understand this now, right?

It's because it's like the AI from her.

I can buy this.

Um, just kind of interesting thinking about like the, the sort of surface part

of this, because it actually will really have a big impact on the market as well.

It's kind of like the technical advances are driving the marketing of this.

I mean, I do think when you, when you look at like some of the initial

reviews of, I don't know, honestly like the pin and rabbit like I remember one

of the one of the scenarios that was being demoed was I think they were I

think he was looking at a car and he was asking a question about it and The

whole interaction took like 20 seconds there and he went through his it was

just showing that he could do the whole thing On his phone in the same amount of

time but the thing that I was thinking about when I was watching that was like

He just did like 50 steps on his phone.

That was awful.

As opposed to just pushing a button and asking a question.

And it was like, it was very clear that the UX interaction of just like, like

asking the question and looking at the thing, was a way better experience than

pushing the 50 buttons on your phone.

But the 50 buttons still won just because it was faster to do 50 buttons than

to, you know, deal with the latency impact of, um, of where we were before.

And so it actually, it reminded me a lot of, Just the way I used to hear,

remember hearing Spotify talk early about the way that they thought about

latency and the things that they did to just make the first 15 seconds of a

song land, um, essentially, so that it felt like, you know, appeal, like a file

that you had on your device, because I think from their perspective, They,

it felt like every time you wanted to listen to a song that was buffering as

opposed to sitting on your device, you were never going to really adopt that

thing because it's a horrible experience relative to just having the file locally.

And so they put in all this work so that it felt the same.

And that wound up being a huge part of how the technology ended up getting and

the product ended up getting adopted.

And, you know, I do think there's a lot of, a lot of stuff we're doing that.

It's almost like, I don't want to say back office, but like just enterprise

processes around how people do things, operational things, but there are plenty

of ways where people are thinking about the way that we do more with like agents

in terms of how that involves like customer experience, whether it's support

interactions, whether it's like bots on the site, you can just Clearly imagine

that that's going to play a bigger role in customer experience going far forward.

And if you feel like every time you ask a question that you're waiting 20

seconds to get a response from this thing.

Like you're just getting the other person on the end of that interaction

is just getting madder and madder and madder the entire time.

Where the more it feels like you're talking to a person

and that they're responding to you as fast as you're talking.

I think the more likely it is that people are going to accept

that as an interaction model.

Um, and so I do think that that latency and like making that feel to you,

like to your point about having a human beings being zero latency, um,

I think that's a necessary condition for a lot of these interaction models.

And so it's going to be super important going forward.

And to me, it's also when I think about the Spotify thing, it's

like, Are people are going to do interesting things to solve for the

first 15 seconds of an interaction as opposed to the entire interaction?

Like, you know, can you get, there was a lot of talk about like

opening, I'm small I want to say like responding with like, sure, or just

like some space filling entry point.

Um, so it like, it could catch up with the rest of the dialogue.

So I think it, I think people will prioritize that a lot

because it don't matter a lot.

I mean, I love the idea that like to save to save cost, basically,

OpenAI is like, for the first few turns of the conversation, we

deliver the really fast model, so it feels like you're really having,

like, a nice, flowing conversation.

And then, basically, once you build confidence, they, like, fall back to,

like, the slower model that has better results, where you're like, oh, this

person is a good conversationalist, but they're also smart, too, right, is, like,

kind of what they're trying to do by, kind of, playing with model delivery.

Um, so.

We got to talk about search, but Chris, I saw you go off mute, so do

you want to do a final quick hit on the question of latency before we move on?

No, I was just coming to come up with what Brian was saying there, and what

you were saying, Tim, I totally agree.

It was always doing this, hey, and then repeat the question, so I wonder

if underneath the hood, as you say, is there's a much smaller classifier

model that is just doing that hey piece.

And then as you say, there's probably a slightly larger model

actually analyzing the real thing.

So I do wonder if there's two small model, a small model and a slightly larger model

in between there for that interaction.

So it's super interesting.

But maybe the thing I wanted to add to that is we don't have that

voice model in our hands today.

We only have the text model.

So I wonder once we get out of the demo environment and then

maybe in a three weeks time or whatever, we have that model.

Whether that's gonna be super annoying every time we ask a

question, it's gonna go, Hey, and then repeat the question back.

So, it's cool for a demo, but I wonder if that will actually be

super annoying in two weeks time.

Alright, so, last topic that we got a few minutes on, uh, and

this is like, Brian's big moment.

So, Brian, get yourself ready for this.

We've hyped this too much.

I mean, show the Chris, you should get yourself ready, cause apparently

Bryan's gonna, you know, you know.

Everyone else can leave the meeting.

Yeah, yeah, take our eyebrows off here with his, uh, with his rant.

So, the, the setup for this is that basically Google announced,

uh, that AI generated overviews will be rolling out to U.

S.

users and then everybody, uh, in the near future.

And I think there's two things that are to set you up, Brian.

I think the first one is, this is what we've been talking about, right?

Like, is AI going to replace search?

Here it is.

You know, here it is consuming.

The preeminent search engine.

So I think it's like, we're here, right?

This is happening.

And then the second one is like, I'm a little nostalgic.

You know, someone who grew up with Google.

Um, you know, I'm like the ten blue links.

You know, like the search engine.

You know, it's like a big part of how I experienced and grew up with the web.

And um, You know, this seems to me like kind of a big shift in how we

interact with the web as a whole.

And so, I do want you to kind of first talk a little about what you

think it means for the market, um, and, uh, and how you think it's going

to change the economy of the web.

So, I follow two communities, I would say, pretty closely online.

I follow the tech community and, uh, pretty closely.

And then I, as a, somebody who works in marketing, I follow my SEOs community.

Um, and they have very different, um, Reactions to, uh, to what's going on.

I think your first question, though, of, um, you know, is this the

equivalent of swallowing the web?

Um, and from the minute, what's funny is from the minute sort of chat GPT

arrived on the scene, people were proclaiming the death of search.

Now, for what it's worth, if you've worked in marketing or on the internet

for a while, people have proclaimed The death of search has like an annual

event for the last like 25 years.

And so, um, this is just like par for the course on, on some level.

But what's interesting to me is that you had this product chat GPT, which is fast

growing consumer product ever a hundred million users faster than anybody else.

And.

What was interesting is sort of like speed run, speed ran the sort of growth

cycle that usually takes years or decades.

I'm like, well, maybe not decades, but like it takes a long time for most

consumer companies to do what they did.

And the interesting thing about that is if it was gonna totally disrupt search,

you would have expected it to show up and happen sooner than it would have

with other products that maybe would have had a slower sort of growth trajectory.

Um, but that didn't happen.

Like as somebody who watches their search traffic super

closely, like there's been no.

chaotic drop of of this, like people have continued to use search engines

and like one of the reasons I think that that happened is because people

actually misunderstood, um, like, like the equivalent of like chat GPT

and Google as competitors, um, with one another, I know Google and open

AI probably are on some level, but I don't know that those two products are.

And the reason I was thinking about that is like, if if chat GPT didn't You know,

within the within basically the time plan we've had so far, uh, disrupt Google.

The question is like, why, why didn't that happen?

And I think you could have a couple different hypotheses for that.

Like one, you could say the form factor wasn't right.

It wasn't text that was going to do it.

It was, we needed Scarlett Johansson on your phone.

And that's the thing that's going to do it.

And so they're leaning into that thought process a little bit.

You could say it was hallucinations, like, oh, the content's just not accurate.

Uh, yeah, right.

So that's a possibility around it.

You could say just like learn consumer behavior.

People have been using this stuff for 20 years.

It's going to take a while to get them to do something different.

You could say Google's advantages in distribution.

So it's like we're on the phone.

We got browsers.

Um, it's really hard.

to, you know, get the level of penetration that we have.

I think all of those probably play some role.

But my biggest belief is that it's actually impossible to separate

Google from the internet itself.

Um, Google is kind of like the operating system for the web.

So to disrupt Google, you actually are not disrupting search.

You have to disrupt the internet.

Um, and it turns out that that's an incredibly high

bar, um, to have to disrupt.

Because you're not only dealing with search, you're dealing with the

capabilities, whether it's banks or, you know, Airlines or, you know, retail,

whatever it is, every single website that sits on the opposite end of the internet.

And it turns out that that's like an enormous amount of capability,

um, that's built up there.

And so I looked at it, I look at that and say like for as much as Like I think

this, this technology is brought to the table, hasn't done that thing yet.

And so because it hasn't done that, there hasn't been some dramatic shift there.

The thing that Google search is not good at though, um, and I think you see it

in a little bit in terms of how they described, you know, What they think

the utility of AI overviews, um, will be is that it's not good at complex

multi part questions of saying, like, if you're trying to plan, if you're doing

anything from, like, doing a buying decision for a large enterprise product

or, like, planning your kid's birthday party, like, you're going to have to do,

like, 25 queries along the way there.

And you just, you've just accepted and internalized 25

like, basically, like, search is one shot, right?

Like, you just say it and responses come back.

So there's no.

Yeah, sorry, go ahead.

Yeah, and so, like, the way I was thinking about LLMs is they're kind

of like Internet SQL, um, in a way, where you can ask this, like, much more

complicated question and then you can actually describe the way that you want

the output of that thing to look it's like, I want to compare these three

products on these three dimensions, go get me all this data, and that

would have been 40 queries, um, at one point, but now you can do it in one.

And search is terrible at doing that right now.

You have to go cherry pick each one of those data points.

But the interesting thing is that that's also maybe the most valuable query to

a user because you save 30 minutes.

And so I think Google looks at that and says, Um, if we seed that particular space

of complex queries to some other platform, like, that's a long term risk for us.

And then if it's a long term risk for them, what it ends up being is a long

term risk for the web, um, I think.

So I actually think it was incredibly important that Google bring this type

of capability into, into the web, even if it ends up being disruptive a little

bit from a publisher's perspective.

Because what it does is at least preserves some of the dynamic we have now, of

like the web still being an important.

thing.

And I hope that continues to your point.

I have like present and past nostalgia.

Yeah, exactly.

So I think it's, I think it's important that it continues to evolve if we all

want the web to continue to persist as like a healthy dynamic place.

Yeah, for sure.

No, I think that's a, that's a great take on it.

And you know, Google always used to say, look, we measure our success based on

how fast we get you off our website.

Right?

And then kind of, Brian, what you're pointing out, which I think is, is

very true, is that like, what they never said was, there's this whole

set of queries we never surface that, you know, you really have to kind

of keep, keep searching for, right?

And like, that's, that ends up being kind of like, uh, the, the, the

search volume of the future that everybody wants to, to capture.

Um, well, uh, so Brian, I think we also had a little intervention

from AI, the thumbs up thing.

We were joking about that before the show.

It's just, yeah.

I didn't know that My ranking for worst AI feature of all time.

Um, so, um, but, um, yeah,

the thumbnail on the, on the video.

That's right.

Exactly.

Um, well, great.

So we've got just a few minutes left Shobhit, Chris, any final

parting shots on this topic?

Sure.

I'm very bullish.

I think, uh, AI overviews.

Have a lot of future as long as There's a good mechanism of feedback incorporating

and making hyper personalized a simple query like I want to go have dinner

tonight Say I tell you I want to look for a Thai restaurant Yeah If you look

if I go on open Open table or Yelp or Google and try to find that there's a

particular way in which I think through it the filters that I Apply are very

different from how Chris would do it.

Right?

So the way I make a decision If somebody's making that decision for me, great.

The reason why TikTok works so much better than Netflix, on an average, I think,

I was listening to a video by Scott, and he mentioned that we spend about

155 minutes a week browsing Netflix.

On an average in the U.

S.

and something of that nature.

It's a pretty exorbitant amount of time.

Versus TikTok has just completely taken that fallacy of choice out for you.

When you go on TikTok, the video that they have made, there's

just so many data points.

A 17 second video averaged 16 minutes of viewing time

across your TikTok engagement.

And you have so many data points coming out of it.

Seven, seven, ten of them.

Every few seconds, right?

So they have hyper personalized it based on how you interact with things, right?

Because they have not, not asking you to go pick a channel,

a choice, this is that nature.

They're just showing you the next, next, next thing in the sequence.

Hence the stickiness.

They've understood the brains of teenagers and, and, and that

demographic really, really well.

I think that's the direction that Google will go into.

It'll start hyper personalizing based on all the content.

If they're reading and finding out where the receipt of my shoes are,

they know what I actually ended up ordering at the restaurant I went to.

So the full feedback loop coming into the Google ecosystem, I think it's going

to be brilliant if they get to a point where they just Make a prediction on

which restaurant is going to work for me, based on everything they know about me.

That's right, yeah.

I mean, the future is they're just going to book it for you, and a car is going

to show up, and you're going to get in.

It's going to take you someplace, right?

Uh, so Chris, uh

They'll send a confirmation from your email.

Yeah, exactly, right.

Uh, Chris, 30 seconds.

You've got the last word.

30 seconds.

Search is going to be a commodity.

And I think as we see the AI assistant era.

How dare you.

Yeah.

But it will be a commodity because we are going to interact with search

via these assistants it's going to be the serial on my phone, which

will be enhanced by AI technology.

It's going to be a Android and Gemini's version on there, we are not going

to be interacting with Google search in the way we do today with browsers.

That is going to be commoditized and we're going to be dealing with

our assistants who are going to go and fetch those queries for us.

So I, I think that's going to be upended and, and at the heart of that is going to

be latency and multimodality as we said.

So, uh, I think they got it or they're going to be disrupted.

Yeah, I was going to say just like if that happens, what's interesting

is that all of the advantage Google has actually vanishes, like, and then

it's an even playing field against every other LLM, which is, you know,

that's a very interesting market situation in that, at that point.

Yeah, I'm going to pick that up next week.

That's a very, very good topic.

We should get more into it.

Um, great.

Well, we're out of time.

Uh, Shobit, Chris, uh, thanks for joining us on the show again.

Uh, Brian, we hope to see you again sometime.

Um, and to all you out there in Radioland, if you enjoyed what you heard, you

can get us on Apple Podcasts, Spotify, and podcast platforms everywhere.

And we'll see you next week for Mixture of Experts.