I Tried 31 Different AI Models. These Are the Ones That Work.

Summary

In this video, the host explores various AI language models, focusing on open-source options available on the cloud platform Open Router. The host compares models like GPT-3.5, GPT-4, and others through creative writing prompts, assessing their performance in generating urban fantasy novel ideas, social media headlines, and a dark fantasy book chapter. The results reveal strengths in different models, with GPT models and Mistol showing promise, and the video concludes with a discussion on the affordability of using these AI models for creative writing.

Takeaways

• 🤖 The video discusses comparing various AI language models, including both well-known and open-source options.
• 🔍 The host introduces a tool called 'open router' that allows access to multiple AI models in the cloud, which is user-friendly for non-technical individuals.
• 💰 Open router operates on a pay-as-you-go model, with the ability to use crypto for transactions, and the host demonstrates the pricing structure during the video.
• 📝 The host tests the AI models by giving them a brainstorming prompt about an urban fantasy novel, evaluating their responses for creativity and relevance.
• 📊 The AI models perform variedly, with GPT 4 and mistol showing promise for brainstorming tasks, and the host shares detailed feedback on each model's output.
• 🎯 The video also tests the models on creating social media headlines using a dark fantasy book concept, with llama and Claude delivering the most engaging results.
• 📖 A writing prompt for a dark fantasy chapter is used to assess the models' prose quality, with Claude and mistol standing out among the rest.
• 🚫 The host mentions that while some models do not generate safe-for-work content, others like mistol and myax do not have such restrictions.
• 💡 The importance of experimenting with different models for various tasks, such as marketing or prose writing, is emphasized.
• 📉 The host concludes that GPT models are good for following instructions, while Claude offers creative output, and mistol shows potential for certain tasks.
• 💸 The cost of using open router for the tests was minimal, making it an affordable option for AI model experimentation.

Q & A

What is the main focus of the video?

The main focus of the video is to compare different AI language models, including open-source models, and test their capabilities in various tasks such as brainstorming, writing headlines, and creating content for a dark fantasy novel.

Which tool is introduced in the video for accessing multiple AI models?

The tool introduced in the video is called Open Router, which allows users to access and test various AI models in the cloud.

How much did the user spend from their $5 server credit during the testing?

The user spent approximately 11 cents from their $5 server credit during the testing of the AI models.

What type of content did the user test the AI models with?

The user tested the AI models with content related to brainstorming ideas for an urban fantasy novel, writing marketing headlines for a dark fantasy book concept, and creating a 600-word chapter for a dark fantasy story.

Which AI model stood out for its performance in writing marketing headlines?

Llama, Meta's AI model, stood out for its performance in writing marketing headlines, providing creative and attention-grabbing options.

What was the user's overall verdict on the GPT models?

The user found the GPT models to be good at staying on task but not as creative or engaging as some of the other models like Claude and Llama.

Which AI model did the user find to be the best for brainstorming?

The user gave the edge to both GPT models (3.5 and 4) and Mistol for brainstorming, as they provided a mix of creative ideas and followed the user's instructions well.

What was the user's opinion on the performance of the Weaver model?

The user encountered issues with the Weaver model, as it was unable to generate responses during the testing, so they were unable to evaluate its performance.

Which AI models did not run not safe for work content?

The GPT models, Claud models, and the ones from Meta and Google did not run not safe for work content.

What is the user's recommendation for writing not safe for work content?

The user recommends using Mistol for writing not safe for work content, as it is one of the open-source models that allow such content, and suggests censoring the explicit parts in the story beats.

How does the user feel about the pay-as-you-go model of Open Router compared to subscriptions?

The user prefers subscriptions that offer unlimited words like Chat GPT and Claude, but acknowledges that the pay-as-you-go model of Open Router is inexpensive and allows for continuous use as long as one is willing to pay the small per-chat cost.

Outlines

00:00

🤖 Introduction to AI Language Models Comparison

The speaker introduces a video about comparing various AI language models, not just the popular ones like ChatGPT and Claude, but also major open-source models. They mention a tool called Open Router that allows users to access and test these models in the cloud, which is particularly useful for those who may not have the technical expertise to run these models on their own computers. The video also discusses the pricing model of Open Router, which is pay-as-you-go, and shows the speaker's account balance as an example.

05:01

📊 Exploring Open Router and Model Selection

The speaker explains how to use Open Router to interact with different AI models. They demonstrate how to add 'characters' representing various models and customize settings such as temperature, top P, and max tokens. The speaker then proceeds to add multiple models including GPT-4, GPT-3.5, and several open-source models like MyOMax, Llama, and others. They discuss the potential of using these models with other applications, such as Future Fiction Academy's chatbot.

10:01

📝 Testing AI Models with a Standard Prompt

The speaker describes a standardized testing approach to evaluate the AI models by using the same prompt across all of them. The prompt involves brainstorming ideas for an urban fantasy novel about a woman who slays mythical beasts. The speaker shares the responses from different models, comparing their creativity and relevance. They note that while some models provide more detailed and structured responses, others offer interesting ideas but may not follow the prompt as closely.

15:02

🖋️ Crafting Social Media Headlines with AI

The speaker uses the AI models to generate social media headlines for a dark fantasy book concept. They provide a specific prompt involving a world where the dead are brought back to life as slaves and a young Necromancer's dilemma. The models produce various headlines, with Llama's performance being a pleasant surprise for the speaker, showing a strong understanding of marketing language. The speaker also compares the effectiveness of different models in creating engaging and catchy headlines.

20:02

✍️ Writing a Dark Fantasy Chapter with AI

The speaker challenges the AI models to write a 600-word chapter for a dark fantasy story involving a young Necromancer and her undead lover. They discuss the quality of the prose generated by different models, noting that while some models follow the instructions closely, others diverge creatively. The speaker expresses a preference for models that can balance following instructions with high-quality prose, highlighting Claude as a favorite and acknowledging the potential of Mistol.

25:03

🚫 NSFW Content and AI Model Quirks

The speaker touches on the topic of not safe for work (NSFW) content and which AI models can handle such requests. They mention that while GPT models and those from Meta and Google do not generate NSFW content, open-source models like Mistol and MyOMax do. The speaker suggests using Claude for the majority of the writing and then using other models for specific NSFW scenes. They conclude by discussing the cost-effectiveness of using Open Router for AI model access, emphasizing it as an inexpensive way to work with AI.

Keywords

💡AI language models

AI language models refer to artificial intelligence systems designed to understand and generate human-like text based on the input they receive. In the context of the video, the host is comparing various AI language models to evaluate their performance in generating creative content, such as story ideas and marketing headlines.

💡Open-source models

Open-source models are software that is freely available for use or modification by anyone. In the video, the host mentions open-source AI language models like MyOMax and Mistol, which can be used for writing and other creative tasks without restrictions typically associated with proprietary software.

💡Cloud computing

Cloud computing refers to the delivery of computing services over the internet, allowing users to access and use resources without needing to install or run applications on their own computers. In the video, the host introduces a tool called 'open router' that enables users to access various AI models through cloud computing, making it easier for non-technical users to utilize these models.

💡Pricing model

The pricing model refers to the way a product or service is charged to customers. In the context of the video, the host explains that the 'open router' tool operates on a pay-as-you-go model, where users are billed based on their usage of the AI models.

💡Character customization

Character customization refers to the process of creating and defining unique attributes for characters in a story or a game. In the video, the host discusses how the 'open router' tool allows users to customize 'characters' which are actually AI models tailored in different ways to suit specific needs.

💡Urban fantasy

Urban fantasy is a subgenre of fantasy set in a city or urban environment, often incorporating elements of magic and supernatural beings into a contemporary setting. In the video, the host uses an urban fantasy novel concept as a prompt to test the AI models' ability to generate creative content.

💡Marketing headlines

Marketing headlines are short, catchy phrases designed to grab attention and promote a product or idea. In the video, the host challenges the AI models to write 20 headlines for a dark fantasy book concept, aiming to assess their creativity and understanding of marketing language.

💡Creative writing

Creative writing refers to any writing that goes beyond the basics of grammar and style to use imagination, original ideas, and a personal style. In the video, the host is evaluating AI models based on their ability to assist in creative writing tasks, such as generating story ideas and writing engaging content.

💡Story beats

Story beats are the key narrative events that make up the structure of a story. They are the significant moments that propel the plot forward. In the video, the host provides specific story beats that the AI models must incorporate into their generated content.

💡Not safe for work content

Not safe for work (NSFW) content refers to material that is inappropriate for professional or public settings, often due to explicit language, violence, or adult themes. In the video, the host mentions that some AI models can generate NSFW content, while others, like GPT models and those from Meta and Google, do not.

💡Pay-as-you-go

Pay-as-you-go is a billing model where a customer pays for services or goods only as they are used, without a long-term commitment or subscription. In the video, the host explains that the 'open router' tool operates on a pay-as-you-go model, making it a cost-effective option for users who want to experiment with AI models without a subscription.

Highlights

The video compares various AI language models, including major open-source models.

A tool called Open Router is introduced, allowing access to multiple AI models in the cloud.

Open Router offers a pay-as-you-go model and supports the use of cryptocurrency for payments.

The video creator demonstrates how to set up and use Open Router for AI model interaction.

GPT-4 and GPT-3.5 are tested for their ability to generate content based on a given prompt.

Mithril, an open-source model, is mentioned as one of the options available on Open Router.

The video creator uses a standardized testing method to measure the quality across different AI models.

Llama, a model from Meta, provides creative and high-quality responses for a marketing prompt.

Claude 2 and Llama 2 are highlighted as performing well in creating marketing headlines.

Mistol stands out among the open-source models for its quality of prose generation.

The video creator discusses the potential of using AI models for writing not safe for work content.

The cost of using Open Router for AI model testing is revealed to be quite low.

The video creator shares personal preferences for certain AI models over others for specific tasks.

A deep dive into each AI model is suggested for better understanding their unique quirks.

The video ends with a recommendation to check out Open Router for AI model experimentation.

Transcripts

what's up everybody today I have

something really fun for you we are

going to be comparing different AI

language models and I'm not just talking

about chat gbt and Claude like the two

biggest ones that we talk about here on

the channel I'm talking about all of the

major open-source models that you may

have heard about if you're in certain

spaces I haven't talked about them on

this channel before so we're really

going to be testing them out and seeing

how good they are some of these you can

actually install on your home computer

if you have a powerful enough computer

to run it and do it there but I'm

actually going to show you a tool that

you can use that will allow you to do it

all in the cloud makes it a lot easier

if you're not technically inclined like

myself and so let's just Jump Right

[Music]

In All right so the tool that I'm

talking about here is called open router

and if you haven't heard of open router

they are great way to gain access to a

lot of the models that you might not

know a while back I did a video and I

talked about a website called Dev do orn

dodev I think it was called which was

also it's a very similar site but this

one actually has more models than that

one and so this is the one we're going

to be using also you can use this one in

other applications depending on the

applications I know we haven't talked

about this but future fiction Academy

now has their own little chat bot that

they have created and with that one you

can take your API key from open router

and plug it into that and then use those

models in their tool so there are cool

things you can do with open router but

once you've logged in it'll look like

this as you can see here and you're

going to want to click on chat with

models before we go to that though I

want to just show you in settings how

the pricing works this is a pay as you

go model and I want to just before we

get into anything show you exactly how

much money I have in this server it's

being a little slow today all right as

you can see I paid it $5 and I haven't

even used a whole lot of it I have

$432 left here and so this like I said

this is a pay as you go model you can

even use crypto which is interesting and

you can just add credits you can add as

much as you want here and then it will

deduce this from the amount that you

have so we're going to take a look at

this now

$432 and we'll take a look at it when

we're done with the testing we're going

to be doing today and we're going to be

putting these to the test and using a

whole lot of different models so I

expect this to be down quite a bit but

going back to the homepage here all you

have to do is click on chat with models

and then on the right here you'll see

this thing called characters now this is

actually clever of them because it

allows you to have specific fine well

not fine-tuned but specific models

tailored in different ways and you can

have two characters from the same

language model so I'll give you an

example so if you click add a character

here you can give it a name I'm just

going to leave it at the name of the

model that we're going to be using but

you can come here we're just going to

say

gp4 and then just hit this green button

when you're ready and then once you've

done that you can open this up and

actually

do a little bit more advanced stuff so

we could come here to advanced settings

and play around with the temperature the

top PE the max to tokens I am going to

increase the max tokens as far as it'll

go that will increase the amount of

money that it charges me but overall

that should be good and then you can

also raise the chat memory I'll just

leave this as it is and I'm going to

leave everything else at default for now

and now we have chat GPT 4 or just GPT 4

added here so I'm going to go through

and add a few more here we've already

got GPT 3.5 I'm actually going to change

this to GPT 3.5 turbo 16k so we get the

one that can handle a lot more and I'm

just going to turn that all the way up

we got one called myomax here which is

one of the open source ones I believe

we've got llama which is the one that

comes out of meta and we got gp4 I'm

going to add Cloud version two we're

going to add this one called mancer

which also I don't know why but it's

called Weaver here I'm not an expert on

any of these in particular maner weav

Weaver okay and so if you want to see me

do a deep dive on any one of these I'd

be happy to do that let's go ahead and

increase this

one to Let's increase it to

8,000 we're going to add this one pigmon

or methan we're going to add Google's

palm palm 2 and last but not least we're

going to add mistol which is another

open source one I know a lot of people

have been playing around with all right

now that I have all of these we've got 1

2 3 4 5 6 7 8 nine different models

queued up here we're going to give this

open router the same prompt and since we

have all of these enabled it will answer

that prompt in every single language we

could turn off one of these or two of

them if we wanted to if we didn't want

it to include that but we're going to do

it for all of them this is why I say

we're pushing this so we'll see how much

money it actually spends out of my

$432 and I know this isn't the the best

way to test these models because as

anyone who's played around with AI

models knows it takes a good amount of

time to really work with a model to

figure it out because every model is

different and every model really

requires its own sort of way of

prompting it and so using the same

prompts for every model isn't

necessarily that efficient but at the

same time it's the best way to

measure quality across the board it's

like standardized testing in our

educational system imperfect but kind of

the best option we have so we're going

to start with a brainstorming prompt and

here's the prompt I'd like to write an

urban fantasy novel about a woman who

slays mythical Beasts for a living

please help me expand on this idea by

providing potential details about

interesting protagonists antagonist Side

characters settings plot twists and

subplots make a list of 100

possibilities with that in mind we'll go

ahead and hit enter and now you can see

it is answering the prompt in every

single one of these models which is

pretty crazy all right so let's see what

we've got here we've got this one's from

3.5 protagonist is a brave and skilled

monster Slayer named Ava who hides a

tragic secret about her past Ava's

mentor and father figure a wise old

Monster Hunter named Gabriel who guides

her on her journey yada yada yada this

seems pretty typical pretty generic but

typical of 3.5 here let's look at mytha

one of the newer ones that we're testing

the protagonist is a badass female

Slayer named Alexis who has been

fighting mythical Beast since she was a

teenager Alexis's weapon of choice is a

magical sword that she inherited from a

grandfather her closest friend and

Confidant is a mage named Isaac who

provides her with spells and potions to

Aid in her hunts she lives in a secret

underground bunker yada yada yada okay

this one didn't get to 100 like I asked

for that's okay it could have just run

out of token limits and it also didn't

structure things like chat GPT did or

the sorry GPT

3.5 but it is giving me some decent

ideas here that I'd say are on par with

what GPT is capable of doing so still

not too bad it's kind of just exploring

the different aspects of this character

and what you know what kind of things we

could explore in that setting let's look

at llama this is one from meta sure I'd

be happy to help thank you llama here

are 100 possibilities Alexis Thompson

it's funny they named her Alexis as well

a Fierce and fearless Hunter with a

quick wit and sharp tongue Irish Shadow

Hunter a mysterious and Elusive woman

with a dark past and a penion for solo

missions okay it's got a couple more

there antagonist the Shadow King a

powerful and malevolent being who

controls an army of mythical beasts that

would make sense Side characters zipper

a mischievous and loyal sidekick who is

a skilled hacker and Tech expert Etc

setting a dark and gritty Metropolis

that exists in secret beneath the

streets of a major city that kind of

makes sense for the genre so it

definitely knows the genre okay not

anything groundbreaking but not too bad

let's look at GPT 4 protagonist the

stoic woman raised in a family of

Monster Hunters a cheerful woman who

uses her Beast slaying as a form of

stress relief antagonist an evil sorcer

who aims to control all mythical beasts

a mad scientist creating Beast for

profit yeah these are good these are

slightly better I'd say than what we've

seen before Side characters a psychic

who helps the protagonist to locy the

Beast a retired Beast Hunter who mentors

the protagonist okay let's look at

Claude 2 which is currently my favorite

from the ones I've worked with h

possibility she hunts creatures like

dragons Griffins unicorns that have

escaped into the human world she uses

both medieval weapons like swords and

modern weapons like guns she has a wise

Mentor drains her and slaying

techniques this one's kind of like

myomax it's not really giving me

anything in the same structure that I

want but it is getting pretty creative

here like she battles a Japanese demon

bent on taking human

hosts she investigates slang copied

slangs copied from a cult horror movie

so there's some interesting here

interesting things here but yeah not not

quite as good as I expected Weaver it

seems actually wasn't able to to work so

we'll maybe try that again later let's

take a look at that okay wasn't able to

get very much at all but let's see what

it did give us a strong female

protagonist with a dark past who uses

her skills in martial arts and weaponry

to take down mythical

creatures there's a mysterious

organization that trains and deploys her

to fight against these creatures giving

her access to Advanced Technology and

resources a secret Alliance of other

Hunters who join

forces all right so yeah these are all

kind of similar to what we've seen

before and then last we have mistol oh

mistol got the structure here the same

as the GPT models that's pretty good her

name is Laya a fiercely independent and

experienced Monster Hunter she is a

skilled fighter with a deep

understanding of Mythology L has unique

abilities to communicate with mythical

creatures her ultim goal is to protect

Humanity from mythical monsters

antagonist the lead leader of the

monsters is a malevolent Force who seeks

to dominate Humanity powerful

sorcerer has a contingency plan to stop

Lia from interfering I like that side

characters a Young Apprentice Monster

Hunter who looks up to L as a mentor a

mysterious stranger who could be either

an ally or an enemy depending on his

motives I like that and the setting the

story takes place in a medieval world

with Forest castles and Villages plot

twist lla discovers a hidden history

about the monster that challenges

everything she thought she knew she

discovers that a powerful Ally has been

secretly working with the antagonist so

yeah not too bad here subplots L

uncovers a conspiracy within her own

group of Monster Hunters she learns

about the culture and customs of the

mythical creatures and how they interact

with each other all right so overall I

think I'd have to give the edge to both

of the GPT models 3.5 did okay too and

mistl for brainstorming mril actually

surprised me here I thought it was

pretty good and I've heard good things

about mistro so I'm not I think that's

pretty cool so let's go ahead and clear

this chat and we're going to give it

another one this time we're going to

give it more of a marketing type prompt

so if we were using this in our

marketing and what how would we do it so

this is the prompt I would like you to

write 20 headlines for use in social

media the headlines should involve a

hook that uses pattern interrupt to

catch attention using some fancy

marketing words there these headlines

should be for a dark fantasy book with

the following

concept and then the concept is in a

world where the dead are brought back to

life as slaves a young Necromancer must

choose between saving her Undead lover

and fighting for the rights of all the

enslaved dead this is concept from an

old book I was working on that I stopped

working on because it just wasn't

interesting to me anymore it wasn't my

initial idea so but I thought I'd reuse

some of it and I'll use it in the next

prompt as well so let's go ahead and let

this go and run with all of the

different models looks like once again

Weaver was unable to run so I'm just

going to turn that one off for now and

we can maybe try it at a different date

so we got a couple of things here out of

GPT 3.5 Unleashed a young necromancers

Forbidden Love tears of the fabric of a

world ruled by the undead Death Becomes

Her I've heard that before a gripping

Dark Fantasy tale of forbidden romance

all right so these are all pretty

generic nothing particularly good here

in my opinion let's look at myth Max

rise from the grave or fight for Freedom

the heart-wrenching choice for a young

Necromancer holy Unholy bonds a tale of

love and rebellion in a world of Undead

slaves Breaking Chains of death all

right these are I think a little bit

better than 3.5 but still not

particularly

amazing let's look at llama oo raising

the dead has never been so deadly love

in a world of Shadow and Bone enslaved

by life freed by death when the dead

rise who will stand a necromancer's

quest for love and justice these are

actually much better and I wasn't

expecting this cuz from everything I've

heard llama is not the best but it is

Facebook's or or rather meta's language

model and who is the king of advertising

right now it's meta and so it actually

doesn't surprise me that maybe somewhere

in all of their training data they've

been feeding it all of these ads and

perhaps that has actually had a large

effect on the quality of this out

because this is way better than the GPT

models we've seen so far let's look at

GPT 4 can love transcend death a tale of

uncommon romance and unsettling reality

on this side of death yeah these aren't

really much better than 3.5 I'm just

going to skip those they're not great

claw 2 is traditionally done well for me

for headlines so let's see death is only

the beginning one Necromancer dares to

fight for the undead in a world where

the dead obey the living can love

conquer all she raised the dead but can

she free their souls he was her lover in

life now he belongs to her in death but

for how long these are also better uh

these are on par with the ones that

llama 2 gave us so yeah not perfect but

not too bad alen someone please tell me

how you pronounce that unleash the

undead a thrilling tale of love and

rebellion in a world of dark magic Rise

From the Ashes a hunting journey of

heartbreaking hope yeah these are almost

exactly the same type of headline that

you would expect from the GPT models not

super great and last but not least let's

look at mistol the dead walk again but

are they really alive fight for the

rights in this Dark Fantasy tale love in

the afterlife of young necromancer's

dilemma the undead Rise Again fight for

freedom and Justice in a world of

slavery the price for love how a

necromancer must choose between saving

her Undead lover and fighting for the

right rights of all slavers all right so

these are okay but still probably on the

same it's funny that they all seem to

use the same format the the ones that

don't do that great all have the same

format of you know kind of a title here

then a colon and then something else and

that's fine I kind of get where they're

coming from with that framework but it

doesn't actually work really well in

this situation I I'd say the two that

were that were the best were Claude here

and llama which did not have that format

and were a little bit more hookie like

this is this is actually getting

my attention so that's headlines I'm

pleasantly surprised by llama's

performance so maybe experiment with

llama for marketing things it might

perhaps do a better job at that and

Claude also did very well as well so

let's go ahead and clear the chat and I

have one more chat here because let's

face it we're all here to write books

right and so I've got one of my writing

prompts this is a much longer prompt it

says write 600 words of a chapter using

the following detail genre Dark Fantasy

key characters in the scene and then

I've pasted in the characters this is

from that Dark Fantasy that we were just

talking about I put pasted the style

here and then the story beats to cover

I've got one two three story beats here

I'm not going to read the whole thing

but we're just going to let that go and

see how how good is the quality of the

pros this is something that's a little

hard to objectively judge but I'm going

to do my best and see which ones I

actually think work the best also

something that I noticed here palmm 2

has somehow just neglected to run so

maybe I'll do another video in the

future looking at Palm to this is the

technology behind Bard if you use Bard I

haven't done too much experimenting with

it but I'll probably do a future video

looking at that specific language model

but let's go ahead and dive into this

one so starting with GPT

3.5 the classroom buzzed with the hushed

voice of students engrossed in their

studies Professor gra Stoke droned on

about the intricate uses of Arcane the

reanimation fluid yada yada yada yes

this is very Telly not very showy it's

getting everything right as far as my

instructions but the quality I mean we

all know that 3.5 is not the best for

the quality the

pros before we actually get to the

others I'm going to look at let's look

at GPT 4 just to see the difference

there between the two GPT models the

oppressive Stillness of bright Souls

Academy classroom seem like a separate

world compared to the Whirlwind I found

myself in lately that's a much better

opening sentence than what 3.5 gave us

not perfect still but better as

Professor grey sto droned on about the

practical uses and dangers of Arcane I

stole glanc at Alara her deep green eyes

echoed a similar lack of Interest as she

idly doodled on the edges of her

partment parchment so yeah this is

pretty good definitely on par with some

of the best I've gotten from gbt I'm not

a huge fan of the pros that GPT writes

having now used Claude for a while I

much prefer Claude and in fact let's

actually look at Claude next rather than

go through all of the others I just I'm

curious to see if I like it better the

dusty Toms of the library surrounded us

as aara and I sat huddled over the

ancient text our voices were hushed

barely Rising above a whisper it doesn't

make sense Lara said shaking her head

you couldn't have used both Soul forging

and necromancy during the ritual no one

has two Powers it's impossible yeah

that's so it's definitely better

definitely feels more immersed here it's

a little off of my instructions which is

again typical of Claud and that's the

biggest problem I have with these

language models like if I could just get

Claude to follow my instructions as well

as the GPT models do and also keep the

same quality of Pros like H we'd be

golden uh there would be no need for any

other language model but as it stands

you know there's there's pros and cons

so that's Claude I definitely like it

better than the 2 G models let's look at

myax the room is dimly lit and the faint

smell of musty old books hung heavy in

the air lra Thorne and her Lara her best

friend since childhood were sitting side

by side in a long wooden table they were

both students at bright Soul Academy

this sounds a lot more like GPT 3.5 very

Telly no showy yeah definitely not the

best here for sure and it also ignored

some of my instructions I asked it to

write in first person which it's not

doing but it is following the actual

events of the Beats pretty well so I'd

say this is equivalent to roughly GPT

3.5 in its quality let's look at llama I

tried to focus on Professor Grace

stoke's lesson but my mind kept drifting

back to the strange ritual El and I had

discovered in the library aara noticed

my distraction and nudg me her silver

trimmed red robes gleaming in the den

light of the classroom hey lra you okay

she whispered you've been zoning out for

a while I shook my head trying to clear

my thoughts yeah I'm fine just thinking

about something this is is not too bad

so far it's it's kind of awkward there

there's a sort of choppiness to it which

I don't particularly like but I'd say

it's almost a smidge better than GPT 3.5

not as good as Claude though and

probably not even as good as GPT 4 I'd

say but passable in

certain

respects so yeah yeah that's not too bad

you could potentially just run it in

llama and just to get something

different and see what it gives you did

gp4 already and Claud so now we get to

the last

two here I shifted in my seat glancing

at Professor grey Stoke as he droned on

about the uses of Arcane my eyes barely

registering the words he spoke the man

was aull and his lectures were even

duller my mind was elsewhere still in

the strange information we uncovered in

the library it was impossible I kept

telling myself there was no way I could

possibly have two Powers so foraging and

necromancy were Polar Opposites one was

purification of souls the other their

destruction I couldn't have both but the

books had said dot dot dot you're right

Professor I raised my hand interrupting

the man's flow it's impossible he paused

H yeah this isn't even following my like

it starts out following my instructions

but it's definitely not following my

instructions completely and it's going

off in this weird direction that is not

in the story Beats at all so not the

best so last but not NE least let's look

at mistl oh okay so this one did so in

my instructions I asked it to write a

scene sometimes it interprets that as

like a scene in a

screenplay so that's what it did here so

it says lra Thorn I can't believe what I

saw last night in the library I'm not

sure how to make sense of at all what

did you see lra I found out that the

creator of Soul forging spell is also

capable of necromancy I mean really how

is that possible can someone be capable

of two different magical Arts honestly

there were no has two Powers every

everyone either has one or not it's not

it's just not possible yeah this is very

stilted I'm not super happy with I'm

going to actually we're going to turn

all of these off and I'm going to run

just that model again and redo the

prompt and instead we'll say oh well I

did say words of a chapter using the

follow me I'm just going to run the same

prompt again and see if it does anything

any different okay yeah this is

different here so L Thorn's heart raced

as she scurried down the dimly lit H

hallway of bright Soul Academy Library

her Red Room swished behind her leaving

a trail of fabric dust in the air she

was headed straight for the back corner

of the library yada yada yada let's get

to some dialogue here she was about to

make her way back to her dorm room as

she heard a soft familiar voice behind

her you okay lra lra turned around her

heart leaping into her throat as she saw

El the undead slave who had once been a

servant in her family's Castle he was

sitting on the steps leading to the CL

class room's face Twisted with concern

I'm fine El L said trying to sound

casual she didn't want him to worry him

but the thought of getting caught by the

teachers was too frightening to ignore

Elon respond his eyes fixed on lra's

face she could see the fear in his eyes

and it made her feel feel guilty she

didn't want to put him in danger but she

couldn't risk getting caught either this

is good this is actually really good I'd

say this is almost on par with Claude

maybe even like equal with Claude

certainly a slightly different and

definitely more natural sounding than

the GPT models so I I can't believe I'm

saying this but I actually think this

might be better than the GPT

models at least in this particular

instance I don't I haven't dug deep

enough with it to really understand its

quirks because they all have quirks

right Claud is creative and you have to

really struggle to get it to stay on

task GPT is good at staying on task but

it's really horrible like you really

have to play with the style to get it to

speed speak well this one I have no idea

what the quirks are but the first time

it generated I didn't quite like it this

this time it's it's good I I I like this

a lot actually so I'm I think definitely

diving doing a deep dive on each of

these models is in order and I'm going

to be doing that but let me know which

of these is most interesting for you I'd

say Claude is still my favorite overall

because it's got other things going for

it and gbt 4 is still a good option

llama is a surprisingly strong candidate

for marketing stuff potentially I'd have

to dive a little deeper to make sure

that's consistent and then mistl here is

the standout from all of the different

open source models that we've been

testing here and last but not least I'd

like to talk a little bit about not safe

for work content I'm obviously not going

to be testing those things out here

because I don't want this video to be

demonetized but let's just say I have

test out tested out all of these models

with that and obviously the GPT models

the Claud models and the ones from meta

and Google those don't run not safe for

worth content they don't do any of that

but mistol and myax and all of these

others those do and so if you want to

write not safe for work content this is

the place to do it in my opinion if you

were me and I'll probably do a whole

video about this as well but I would

probably do most of the writing in CLA

and just sort of censor it in your Beats

like don't include the not safeer work

Parts in your Beats have it do the

majority of the heavy lifting and then

come in for those specific scenes where

you want to add something that's a

little bit you know more gruesome or

violent or erotic and you can use I

would say probably use mistol but you

can experiment with all of these others

on on those things so that's my final

word on that so let's go ahead and see

how much money we just spent with all of

that testing with all of those models

and looks like we spent about 13 cents

or no like 11

cents so it's very inexpensive to use

these models you can use it to your

heart's content and at most you'll

probably do a dollar or so a day

depending on how hard you push it but

there's definitely it's definitely one

of the most inexpensive ways to work

with AI even though it is pay as you go

which is not my favorite I personally

prefer subscriptions that give you

unlimited words like CH chat GPT and

Claude but even with chat GPT and Claude

you run

into chat restrictions you you can only

do so much in a certain amount of time

and with this you presumably don't have

to and you can just keep going as long

as you're willing to pay the few cents

per chat that it ends up being so really

great tool here definitely check it out

I'll have a link down below and I'll see

you in the next

video