Build with Claude 3.5 to win $30,000 | OpenAI Voice Mode Delayed | ElevenLabs Speech to Speech Rocks

Summary

The video discusses the impressive coding capabilities of Cloud 3.5 Sonet, highlighting its recent achievements in the coding arena. It also covers the 'Build with Cloud' contest by Anthropic, offering $30,000 in API credits for creative app development using the Anthropic API. Additionally, new features of Cloud 3.5, such as custom knowledge bases called 'projects' and custom instructions for specific outputs, are explored. The video also mentions 11 Labs' text-to-sound API and the potential for integrating it into contest submissions.

Takeaways

• 🌐 Cloud 3.5 Sonet has become the top coding AI, surpassing GPT 40 Gemini 1.5 Pro and its predecessor CLA 3.
• 🔍 The script discusses the impressive coding and troubleshooting abilities of Cloud 3.5 Sonet, with a noticeable leap in its capabilities.
• 📊 The voting scores for AI models have not been updated recently, with the last update mentioned as June 23rd.
• 🎉 Anthropic is hosting a 'Build with CLA June 2024' contest, offering $30,000 in API credits for the best apps using the Anthropic API.
• 🏆 The contest requires building a creative app that effectively uses the Anthropic API, with submissions due by July 10th at 12:00 p.m. Pacific.
• 🤖 Alex Albert from Anthropic is sharing insights on Claude AI, hinting at rapid developments and updates in the field.
• 📝 New UI features have been added to CLA, including a sidebar for chat organization and the ability to create custom knowledge bases called 'projects'.
• 👥 Team members can collaborate on projects within CLA, sharing custom knowledge bases and files specific to their team.
• 📝 Custom instructions can now be given to CLA to tailor its responses, such as using a specific language or providing a chain of thought reasoning.
• 🎁 The script mentions the potential of using 11 Labs' text-to-speech API to create sound effects for projects in the CLA contest.
• 🤝 Multiplayer, a company focusing on shared computing experiences, is joining OpenAI to explore new collaborative features.
• 🎙️ 11 Labs has introduced a new feature allowing users to dictate the inflection of AI's voice, enhancing the realism of generated audio.

Q & A

What is the current status of Cloud 3.5 Sonet in the coding arena?

Cloud 3.5 Sonet is currently ranked number one in the coding arena, surpassing other models like GPT 40 Gemini 1.5 Pro and its larger predecessor CLA 3.

What is the significance of the 'build with Cloud' June 2024 contest?

The 'build with Cloud' contest is an initiative by Anthropic where participants can win $30,000 in Anthropic API credits by building and sharing an app that uses Cloud through the Anthropic API.

What are the requirements for submitting a project to the 'build with Cloud' contest?

To submit a project, participants need to build an app that demonstrates creativity and effectively uses the Anthropic API. The project should be submitted through a linked Google form, and participants are encouraged to post their project on X and tag Alex or Anthropic AI.

What is the deadline for the 'build with Cloud' contest?

The deadline for submitting projects to the 'build with Cloud' contest is July 10th at 12:00 p.m. Pacific.

How many winners will be selected in the 'build with Cloud' contest, and what will they receive?

Three winners will be selected based on creativity, impact, usefulness, and implementation. Each winner will receive $10,000 in API credits.

What new UI features has CLA added to improve user experience?

CLA has added a sidebar for organizing chats and a feature called 'projects' that allows users to create custom knowledge bases with their own files, documents, and code. This enables better organization and collaboration on projects within a team.

What is the purpose of the custom instructions feature in CLA?

The custom instructions feature allows users to give specific directives to CLA, such as responding with a particular accent or providing a detailed step-by-step explanation, to tailor the AI's responses to their preferences.

What is the role of 11 Labs in the context of the Cloud build with Cloud competition?

11 Labs provides an API that can be used to generate sounds, such as an ocean breeze or a video game power-up sound, which could be integrated into projects for the Cloud build with Cloud competition.

What is the significance of the late June Chad bot in the AI arena?

The late June Chad bot is a model that has appeared in the arena rankings, potentially indicating that it is a new model being tested by Open AI, similar to how GPT 40 was previously tested.

What is the new feature introduced by 11 Labs that allows users to control the inflection of the AI's voice?

11 Labs has introduced a feature that enables users to dictate the inflection of the AI's voice by speaking aloud. This allows for more natural and varied speech patterns in the AI's responses.

What is the potential application of 11 Labs' new feature in the context of the Cloud build with Cloud competition?

The new feature from 11 Labs could be used to enhance the user experience of apps built for the competition by providing more natural and engaging voice interactions.

Outlines

00:00

🏆 Cloud 3.5 Sonet's Coding Mastery and the Build with CLA Contest

The script discusses the impressive coding capabilities of Cloud 3.5 Sonet, which has surpassed other models like GPT 40 and Gemini 1.5 Pro in the coding arena. It highlights the model's ability to add features and troubleshoot effectively. The speaker mentions the 'Build with CLA' June 2024 contest by Anthropic, offering $30,000 in API credits for the best apps using the Anthropic API. The contest encourages creativity and effective use of the API, with a submission deadline of July 10th. The summary also touches on the new UI features of Cloud 3.5, such as custom knowledge bases called 'projects' and custom instructions for tailored responses.

05:02

🎮 Innovative Features and the Potential of Cloud 3.5 in Gaming

This paragraph delves into the new features of Cloud 3.5, such as the ability to code a sound generator using 11 Labs API, which can create various sound effects from text descriptions. It also discusses the potential application of these features in the 'Cloud build with Cloud' competition, suggesting the use of APIs for innovative projects. The script mentions the addition of a 'late June Chad bot' in the arena rankings, hinting at possible testing by OpenAI. Furthermore, it explores the concept of 'multi', a company joining OpenAI to work on inherently multiplayer desktop computers, indicating a shift towards more collaborative computing environments.

10:05

🎙️ 11 Labs' Speech Inflection Customization and Audio Editing

The final paragraph focuses on 11 Labs' new tutorial for their text-to-speech feature, which allows users to customize the inflection of the AI's voice by speaking the desired tone aloud. Despite some UI clunkiness in the beta version, the functionality is praised for its potential to enhance the expressiveness of AI-generated speech. The speaker shares their experience with the feature, demonstrating how to correct inflection and add natural elements like laughter to make the AI's voice sound more realistic. This feature could be particularly useful for content creators and developers looking to integrate more nuanced voiceovers into their projects.

Keywords

💡Cloud 3.5 Sonet

Cloud 3.5 Sonet is a coding AI model mentioned as being superior to its competitors like GPT 40 Gemini 1.5 Pro and its predecessor CLA 3. It represents a significant leap in AI coding abilities, as it can add features and troubleshoot effectively. The script discusses its current standing as the top model in the coding arena, indicating a strong advancement in AI's capability to understand and generate code.

💡Coding Abilities

Coding abilities refer to the capacity of an AI to write, understand, and troubleshoot code. In the context of the video, Cloud 3.5 Sonet's coding abilities are highlighted as being impressive, suggesting that it can perform complex coding tasks with ease, which is a central theme of the video discussing AI advancements.

💡Anthropic API

The Anthropic API is a service provided by the company Anthropic, which allows developers to integrate AI capabilities into their applications. The script mentions a contest involving the use of the Anthropic API to build and share apps, emphasizing the practical application of AI in software development.

💡Build with CLA June 2024 Contest

This is a specific contest mentioned in the script where participants are encouraged to build and share applications using the Cloud through the Anthropic API. The contest is a way to promote the use of AI in app development and offers a prize of $30,000 in Anthropic API credits, which is a significant incentive for developers to engage with the technology.

💡Creativity

Creativity in the context of the video refers to the ability of AI to generate novel and original ideas or solutions. The contest criteria emphasize creativity, indicating that the AI's potential to produce innovative applications is a key aspect of the video's discussion on AI's evolving capabilities.

💡Custom Knowledge Bases

Custom knowledge bases, or 'projects' as mentioned in the script, are personalized collections of files, documents, and code that AI can access to provide more informed responses. This feature represents an advancement in AI's ability to contextualize information and is an example of how AI can be tailored to specific user needs.

💡Custom Instructions

Custom instructions are specific directives given to AI to shape its responses. In the video, it is mentioned that users can give custom instructions to Cloud 3.5 Sonet to achieve particular outcomes, such as adopting a specific accent or providing detailed reasoning in responses, showcasing the flexibility of AI to adapt to user preferences.

💡Eleven Labs API

Eleven Labs API is a service that provides text-to-speech capabilities and other AI voice features. The script discusses how Cloud 3.5 can use this API to generate sounds, such as an ocean breeze or a video game power-up sound, demonstrating the integration of different AI services to create interactive experiences.

💡Multiplayer

The term 'multiplayer' in the script refers to the concept of collaborative computing, where multiple users can interact with an application simultaneously. The company Multi is joining OpenAI to explore this concept further, suggesting a future where operating systems facilitate collaborative user experiences akin to multiplayer games.

💡Speech-to-Speech Synthesis (STS)

STS is a feature that allows users to input speech with specific inflections, which the AI then replicates in its own voice output. In the script, it is demonstrated as a way to add natural-sounding inflections to AI-generated speech, enhancing the realism and expressiveness of AI communication.

💡UI Features

UI features refer to the visual elements and interactive components of a software application. The script mentions new UI features added by Cloud and its team, such as a sidebar for chat organization and project management, which aim to improve user experience and productivity when using the AI service.

Highlights

Cloud 3.5 Sonet has become the top coding AI, surpassing GPT 40 Gemini 1.5 Pro and its predecessor CLA 3.

There has been a significant leap in Cloud 3.5's coding, feature addition, and troubleshooting capabilities.

The number of votes for Cloud 3.5 has not increased as expected, indicating potential for growth.

Anthropic and CLA 3.5 have been active, with updates not reflected in the latest scores from June 23rd.

Alex Albert from Anthropic has been sharing valuable insights on Claude AI.

Anthropic is hosting a 'Build with CLA' contest in June 2024, offering $30,000 in API credits.

The contest requires building and sharing an app using the Anthropic API, with a submission deadline of July 10th.

Three winners will be selected based on creativity, impact, usefulness, and implementation, receiving $10,000 in API credits each.

Participants are encouraged to use Claude's API to create something simple, even if they are not professional developers.

Watching the submitted projects can provide insights into the diverse applications of Claude's capabilities.

Hume AI, an empathic AI voice, is being featured again, indicating advancements in emotional AI.

Concerns are raised about the rapid progress of AI and its potential impact on society.

CLA has introduced new UI features, including a sidebar for chat organization and custom knowledge bases called 'projects'.

Projects allow for the creation of custom knowledge bases with files, documents, and code accessible within chats.

Custom instructions can be given to Claude to tailor responses, such as language preference or reasoning steps.

Open AI's voice mode is experiencing delays in rollout, with an Alpha group starting to test it.

A new model, 'Late June Chad bot', has appeared in the arena rankings, possibly indicating Open AI's testing of a new chatbot.

Cloud 3.5 can now code a sound generator using 11 Labs API, which offers text-to-sound features.

Multiplayer, a company focusing on shared computing experiences, is joining Open AI to explore new collaborative possibilities.

11 Labs has introduced a new feature allowing users to dictate inflection and tone for AI-generated voices.

The tutorial showcases the capability to correct AI voice inflection through speech-to-speech dictation.

Transcripts

so at this point no doubt you've heard

that cloud 3.5 Sonet is good very good

it's now number one in the coding Arena

beating out GPT 40 Gemini 1.5 Pro even

beating out its larger predecessor CLA 3

Opus AGI rolls around only once

subscribe I've tested its coding

abilities and can't help it be pretty

impressed there does seem to be a very

strong Leap Forward in its ability to

code a to add features to troubleshoot

in fact as you can see here there's not

as many votes that has been thrown here

so over time as this improves as this

number increases I wouldn't be surprised

if this difference between GPT 40 and

Cloud 3.5 if it continues growing the

scores haven't been updated yet so it

looks like the last update was on June

23rd but if you've been following along

if you've been interested in what

anthropic and CLA 3.5 have been up to

this is a great follow Alex Albert I'll

leave a link in the description so he's

working at anthropic on Claude Ai and

he's been dropping some knowledge bombs

as of late the latest one is this this

was posted within this hour they're

announcing the build with claw June 2024

contest and they're giving out

$30,000 in anthropic API credits all you

need to do is build and share an app

that uses Cloud through the anthropic

API the contest is pretty simple build

an app that demonstrates creativity and

effectively uses the anthropic API

submit your project through the linked

Google form and they encourage you to

post your project on X as well as tag

themselves so tag Alex or anthropic Ai

and you have until July 10th at 12:00

p.m. Pacific so two weeks to create and

submit it and they're going to be

selecting the top three projects based

on creativity impact usefulness and

implementation and each of the three

winners will get $10,000 in API credits

oh boy I'm pretty sure Pini already jail

broke lot and uh is up to no good with

it but if this is something that you're

interested in I encourage everybody to

try it out I would not even worry if you

necessarily win the contest or not I

think the practice of creating something

simple perhaps even using Claud to

create it like have it type up the code

for you again you don't have to get

complicated if you're a God to your

developer sure go for it if you have

coded before just starting out doing

something very simple that uses Claude

specifically their API to to produce

something and just submit it but even if

you don't care to do that at the very

least watch what everyone else submits

they're going to be tagging Alex and

anthropic API and anthropic AI so that

you're able to see kind of what projects

people are building I would not be

surprised if we saw some pretty insane

things come out of this you might recall

Hume we've covered it briefly on his

channel is your empathic AI voice and

I'm so happy to be covering it once

again hearing the pilation in your voice

just makes me smile too wow whatever it

is you're so excited about I can't wait

to hear all about it your happiness is

truly infectious fill me in what's got

you so over the Moon looks like Hume AI

is detecting notes of ecstasy in my

voice but if I'm being perfectly honest

I am getting quite concerned about how

good Claud is getting and perhaps a hint

of unease there I totally get it the

rapid progress of AI can be a little

unsettling at times I want to win that

contest I will go to any length to do so

I hear the Steely determination in your

voice you're really going all in to win

this contest huh and maybe we can find a

way for you to pursue Victory without

compromising your principles that's what

do you think so basically they pick up

on your emotions and kind of match your

emotional tone so to speak and when they

ran anthropics Cloud 3.5 Sonet model

through their evaluation system for

Creative ability surprise surprise it

emerged as the new leader clot 3.5 Sonet

bow Gemini Ultra GPT 4 Turbo and all the

other models now on top of this CLA and

the team have been very busy adding some

new UI features that might indeed be

very helpful a sidebar to organize some

of the chats that you've had but much

more importantly this so so this is the

referred to it as projects so you're now

able to create custom knowledge bases

called projects with your own files

documents code Etc when you start a new

chat within a project Claud has access

to all that info and on the team plan

you can share and collab on all these

projects with your teammates so this is

similar to the custom gpts that openi

was rolling out with the one difference

is this is just for your teammates so

you're able to share it on the team plan

but it looks like right now you're not

able to do so you know to just anyone to

to the public they've added custom

instructions so if you need to give some

specific things to Claud to keep in mind

as it answers your questions like for

some reason responding with a thick New

York accent why but okay well at least

it's not the Golden Gate Bridge anymore

but this works really well for

situations where you just prefer certain

outputs so for example if you're coding

you can always ask for for the specific

language that you want or you can always

do Chain of Thought reasoning by asking

you to think through something step by

step a lot of people put you know

respond with short answers and to the

point text I would test to see if that

uh May reduce the quality of the ANS

sometimes it's sometimes better for the

models to think through their answers

and they do that by you know putting

tokens on the screen so longer answers

may be better than shorter you got to

kind of test it out it really depends on

what you're using it for this person

seems blown away that these features

that they announced are available right

now I thought you're supposed to say

available in the coming weeks and then

keep delaying them for months and months

which unfortunately that is kind of the

case with open AI right now they're

delaying the roll out of their voice

mode that they've demoed um during their

spring update now they did say they're

going to be rolling out to a small Alpha

group of testers and we are beginning to

see that it is indeed happening we are

beginning to you know to see people

posting videos of them messing around

with it and using it here's an example

of that this is legitimate as far as I

can tell uh we know it's rolling out

account seems legit but of course we

can't know for sure but here's the demo

even though it's summer there's this

refreshing coolness in the air that just

makes you want to smile and take a deep

breath of that crisp invigorating Breeze

the Sun's shining but is that this

lovely gentle warmth that's just per

perfect for light jacket or cozy St so

May the odds be ever in your favor and

all that meanwhile another Secret model

is popping up in the arena rankings it's

called the late June Chad bot so last

time we saw this it was open AI testing

the release of GPT 40 they had some

naming convention like I'm a good gpt2

bot another one was something like I am

also a good gpt2 bot I forget exactly

but it was kind of similar to this it

was revealed later that it was in fact

GPT 40 so is this opening eye testing

their late June chatbot we shall see

coming back to Cloud

3.5 it's able to code up a sound

generator now now it uses 11 Labs API

which 11 Labs is the famous kind of AI

voices they have a lot more features

that are rolling out that we'll talk

about in a second but one of them is

text to sound so you can type in things

like an ocean breeze or bang or a video

game PowerUp sound and it'll generate

that effect and play it for you so again

if you're building something for that

cloud build with Cloud competition this

might be potentially something that you

use maybe use an API to hook into 11

Labs produce a voice produce some sound

effects Etc another interesting thing is

this company multi or multi multiplayer

is joining open Ai and they're asking

what if desktop computers were

inherently multiplayer what if the

operating system placed people on equal

footing to apps so this company was you

can think of it kind of was just a

remote computer access company kind of

but now they're joining opene to work

with openi to do that plus AI we're not

100% sure exactly what they're doing but

it's going to be interesting to find out

so this is multi and they're saying

screen share down right and everything

else too so they have features like

basically screen sharing multiplayer

right so communicate faster with shared

cursors drawing using screen share from

up to 10 people at a time and various

other tools for advanced screen sharing

and remote desktop assistance Etc so I

have a few ideas of what I'm going to

attempt to build for this CLA contest I

don't expect to win it's probably going

to be pretty simple but I do view this

as an opportunity to kind of uh just

learn something new here's Ethan mik

saying that you know if you ask Claud to

create something like a snake game it

has a lot of sort of examples of that

and its training data but if you give it

something weird and out of its sort of

uh data distribution it seems like it

does a pretty good job of that here the

prompt was I need you to create a

simulation where I am a Lighthouse

Keeper in a cosmic Lighthouse and I have

two beams one that attracts space

leviathon and the other repels them and

there's also gravity to take into

account make it fun and make the

graphics good and also add lower and

make it have a goal and this was the

sort of final uh output of that so with

that said good luck to all that are

attempting this we will be watching in

anticipation there's a few more bits of

news that I wanted to play from 11 labs

they created a short tutorial on how to

use a brand new feature that I've tested

and I got to say it's it's interesting

it's still not perfect but I'm pretty

excited let me show you what I'm talking

about so the UI is a bit clunky I think

they just launched it it's still in beta

there's like little issues with it so

I'm not quite getting uh what I want out

of it but most of it I think is just the

UI bugginess but the functionality seems

like it's getting pretty good it allows

you to put whatever inflection you want

Into The ai's Voice by just speaking it

out loud so instead of that kind of

monot Voice or a random inflection you

can specify here's an example you

clearly don't know who you're talking to

so let me clue you in I am the danger I

am the one who knocks I might post a

more kind of involved tutorial a little

bit later I had some difficulty doing

this and that's just the UI issues but

here's one of the people at 11 Labs kind

of showcasing how this would work if

everything's working properly again I

was able to get the functional to work I

was just limited by I feel like some of

the UI features with that said my name

is Wes rth and thank you for watching so

let's start fixing some of these clips

um the first thing I want to draw your

attention to is this clip right here

where he said he'll stop it nothing to

avoid them I noticed a strange like

question mark at the end of that

sentence that just doesn't fit so let's

have a quick listen again you'll stop at

nothing to avoid them and now so we

obviously don't like that we want to

change that you could just keep clicking

generate audio and he might say it in a

different way let's hear how he says it

this time he'll stop at nothing to avoid

them so he seems to be in this pattern

where it's generating over and over in

the way that we don't like it to so

that's where speech to speech comes into

play and it's very effective so I'm

going to click on the clip I'm going to

scroll down here on the bottom right

where it says dictation that's speech to

speech and then I'm going to click the

microphone icon and then start speaking

and Performing the way I want it to be

said exactly and it's going to match

that um to a t so let's go ahead and try

that he'll stop it nothing to avoid them

once I'm done I hit stop and I'm going

to hit generate audio STS give it just a

moment there to generate the new audio

that I just spoke I'm going to move

these clips shift click them and drag

them over just so we have some room on

the timeline maybe a little bit more and

now let's have a listen he'll stop it

nothing to avoid them perfect and he

spoke it the exact way that I input it

haha that's a good one okay obviously

that doesn't sound realistic so let's go

ahead and change that I'm going to click

on the clip and I'm going to add a

little bit of laughter to the beginning

of that to see if it sounds a little bit

more

natural that's a good one speaking of

numbers I tried to count all the stars

last

night that's a good one speaking of

numbers I tried to count all the stars

last night I think that sounds a lot

better that sounds pretty good pretty

pretty

pretty pretty good