03 - Do you know what A/B testing is?

Summary

In this episode of 'Convert on Shopify,' host Pancho Mandiola introduces A/B testing, explaining its importance for optimizing e-commerce websites. He shares his personal experience transitioning his cosmetics brand to e-commerce and highlights how platforms like Shopify helped him succeed. Pancho emphasizes that A/B testing is key for improving site performance by experimenting with different versions of elements like headlines, buttons, and layouts. He encourages entrepreneurs to adopt a culture of experimentation to achieve better conversions and business growth, while also hinting at future episodes with expert guests.

Takeaways

• 📈 A/B testing is a method of comparing two versions of a webpage or app against each other to determine which one performs better.
• 🌐 The podcast aims to educate listeners on A/B testing and conversion optimization strategies, featuring experts from the field.
• 🚀 The host, Pancho Mandiola, shares his personal journey of entering e-commerce and the challenges he faced, including the importance of learning from failures.
• 🛠️ Early e-commerce ventures often failed due to a lack of understanding of UX/UI and the absence of a technical background, highlighting the need for learning and adaptation.
• 🌟 Shopify is highlighted as a platform that allows entrepreneurs without a technical background to create and manage e-commerce stores effectively.
• 📊 A/B testing is crucial for growth as it helps businesses to make data-driven decisions and optimize their online presence for better conversion rates.
• 🔍 The podcast will explore the importance of having a culture of experimentation and data within organizations to drive continuous improvement.
• 🎯 A/B testing involves creating two versions of an element on a website and measuring which one performs better, providing certainty for investment in marketing and traffic generation.
• 📝 Examples of elements that can be tested include headlines, button colors, images, and calls to action, emphasizing the importance of detail in optimizing user experience.
• 💡 The ultimate goal of A/B testing is to refine and optimize website elements to increase conversions, which can lead to higher sales and a more successful online business.

Q & A

What is the main topic of the podcast episode?

The main topic of the podcast episode is A/B testing, with a focus on its importance and application in e-commerce and conversion rate optimization.

Why did the host start exploring A/B testing?

The host started exploring A/B testing because they realized the importance of having a culture of experimentation and data-driven decisions in their organization, especially after experiencing the limitations of not having a technical background in e-commerce.

What was the host's initial experience with e-commerce?

The host's initial experience with e-commerce involved starting a natural cosmetics brand and facing challenges with developing an online store due to a lack of experience in UX/UI and e-commerce.

Why did the host switch from developers to using Shopify for their e-commerce platform?

The host switched to Shopify because it was a more accessible platform for someone without a technical background, allowing them to create an online store with pre-built templates and without the need for coding knowledge.

How did the host's e-commerce business perform after transitioning to Shopify?

After transitioning to Shopify, the host's e-commerce business became very successful, eventually reaching a million dollars in annual sales, outperforming their physical retail shops and wholesalers.

What is the 'fold' in the context of a website's homepage?

The 'fold' refers to the imaginary line on a website's homepage that separates the visible content before a user scrolls down. It is important because most first-time visitors will only see the content above this line.

What elements on a website's homepage can be tested using A/B testing?

Elements that can be tested using A/B testing include the banner, headline, call-to-action button text, background color, images, and the menu on the homepage.

What is the purpose of using a plugin like Convert for A/B testing?

A plugin like Convert is used for A/B testing to measure and analyze the results of different versions of a webpage element, allowing the business to determine which version performs better.

Why is it important for businesses to have a culture of experimentation?

Having a culture of experimentation is important because it allows businesses to make data-driven decisions, optimize their websites, and improve conversion rates, ultimately leading to better performance and sales.

What are the potential benefits of A/B testing for an e-commerce business?

The potential benefits of A/B testing for an e-commerce business include increased conversion rates, better user experience, more efficient use of marketing budgets, and the ability to make informed decisions based on data.

Outlines

00:00

📈 Introduction to A/B Testing in E-commerce

In the first paragraph, Pancho Mandiola introduces the topic of A/B testing within the context of e-commerce. He sets the stage for the podcast by explaining that the initial episodes will focus on foundational concepts, which are essential for understanding more complex discussions with future guests. These guests will include experts from the Convert team, conversion rate optimization agencies, and entrepreneurs who have successfully implemented A/B testing strategies. Pancho emphasizes the importance of learning the basics to have meaningful conversations with these experts without needing to pause for clarifications on terminology. He also shares a personal anecdote about his early foray into e-commerce with a natural cosmetics brand, highlighting the challenges he faced in developing an online store due to a lack of experience in UX/UI and e-commerce.

05:03

🛠️ Transition from Traditional Retail to E-commerce

In the second paragraph, Pancho recounts his journey from traditional retail to e-commerce, noting the success of his small retail shop and wholesale business. He observed the shift towards e-commerce in more developed countries and decided to prepare for the same trend in his home country of Mexico. His initial attempt at e-commerce involved hiring developers to build a website, which resulted in failure due to inexperience with UX/UI and e-commerce on both his part and the developers'. This experience led him to seek better solutions and eventually learn about platforms like Shopify, which were more accessible for non-technical entrepreneurs.

10:04

🚀 Success with Shopify and the Need for A/B Testing

The third paragraph details Pancho's success with Shopify, which allowed him to create an online store without a technical background. He emphasizes the importance of templates in Shopify that helped him avoid the pitfalls of a blank canvas and the challenges of designing from scratch. His online store became a million-dollar business, outperforming his physical retail shops and wholesalers. However, Pancho points out that with growth came the necessity to learn about A/B testing. As the business expanded, he realized the importance of testing and optimization to continue improving the online store's performance and to make informed decisions about marketing investments.

15:06

🧪 Understanding A/B Testing and Its Benefits

In the final paragraph, Pancho defines A/B testing as running experiments on a website to measure the effectiveness of different elements, such as headlines, images, or call-to-action buttons. He explains the process of splitting traffic between a control version and a test version to determine which performs better. Pancho stresses the importance of moving from assumptions to evidence-based decisions, which A/B testing facilitates. He also touches on the broader goal of fostering a culture of experimentation and data-driven decision-making within organizations. The paragraph concludes with a call to action for listeners to embrace A/B testing and to look forward to more detailed discussions in future episodes.

Keywords

💡A/B Testing

A/B Testing, also known as split testing, is a method of comparing two versions of a webpage or other user experience to see which performs better. It's a fundamental concept in the video, as it's the main topic being discussed. The speaker uses the example of testing different headlines on a website's banner to illustrate how A/B testing can help determine which version leads to more engagement or sales.

💡Conversion Rate Optimization (CRO)

Conversion Rate Optimization refers to the process of optimizing web pages to increase the percentage of visitors that convert into customers, sign up for a newsletter, or take any other desired action. In the script, the speaker mentions having guests from CRO agencies, indicating that A/B testing is a tool used within CRO to improve the effectiveness of a website.

💡E-commerce

E-commerce involves buying and selling goods or services using the internet, and it's the industry focus of the video. The speaker shares their personal journey of starting an e-commerce business, highlighting how A/B testing became relevant to their online store's success.

💡UX/UI

UX stands for User Experience, and UI stands for User Interface. Both are crucial in web design, as they relate to how users interact with and perceive a website. The speaker mentions a lack of experience in UX/UI as part of the reason for their initial failure in e-commerce, emphasizing the importance of these fields in creating effective websites.

💡Technical Background

A technical background refers to having knowledge in areas such as coding, web development, and software engineering. The speaker contrasts their initial lack of technical expertise with their later success using platforms like Shopify, which require less technical knowledge to operate effectively.

💡Platform

In the context of the video, a platform refers to the underlying technology that supports a website, like Shopify. The speaker discusses how using Shopify allowed them to bypass the need for a technical background and focus on growing their e-commerce business.

💡Optimization

Optimization is the process of making something as effective, efficient, or functional as possible. In the video, the speaker talks about using A/B testing for optimization purposes, aiming to improve the conversion rates of their online store.

💡Experimentation

Experimentation in this context refers to trying out different elements or strategies to see which works best. The speaker advocates for a culture of experimentation within organizations, using A/B testing as a method to test hypotheses and improve website performance.

💡Control Version

The control version is the original version of a webpage or user interface element that is being tested against a variant in A/B testing. The speaker explains that in A/B testing, the control version is compared to another version to measure which is more effective.

💡Banner

A banner in web design is a graphical element used to display important information or calls to action. The script uses the banner as an example of an element that can be tested through A/B testing, such as changing the headline, colors, or call to action button.

💡Fold

The fold refers to the portion of a web page that is visible without scrolling down. The speaker mentions that more than half of the visitors might not scroll past the fold, making the content above it crucial for first impressions and conversions.

Highlights

Introduction to the concept of A/B testing in the context of e-commerce and Shopify.

The importance of understanding A/B testing for entrepreneurs and business owners.

The host's personal story of entering e-commerce and the challenges faced with website development.

The shift from traditional retail to e-commerce and its impact on business operations.

The failure of the initial e-commerce venture due to lack of UX/UI and marketing knowledge.

The transition to Shopify as a platform for non-technical entrepreneurs.

How templates in Shopify helped in creating an effective online store.

The success story of a natural cosmetics brand reaching a million-dollar business through e-commerce.

The necessity of learning A/B testing for growth and optimization of an online business.

Defining A/B testing as a method to run experiments and measure their impact on a website.

The significance of the homepage and 'above the fold' content in user engagement.

Examples of elements that can be tested through A/B testing on a website's homepage.

The process of conducting an A/B test using a plugin like Convert and analyzing the results.

The goal of A/B testing to move from assumptions to data-driven decisions for business growth.

The importance of having a culture of experimentation and data within organizations.

Invitation for listeners to engage with the podcast by suggesting topics or referring interested friends.

Transcripts

[Applause]

[Music]

hello my name is pancho mandiola welcome

to a new episode of convert on shopify

so today we're going to be talking about

what is a b testing

and as we said in the previous episodes

these first ones are all about getting

to know the concepts that we're gonna be

using all along the way once we bring

the big guns because yes we're gonna

have some awesome guests here in this

podcast obviously we're gonna have

people from the convert team and they

believe me they are very knowledgeable

when it comes to a b testing experiments

and all all about conversion so we're

gonna have a couple of them we're gonna

also gonna have people that uh have

their own agencies specifically

conversion rate optimization agencies so

these are like the

the superheroes in the world of

e-commerce

we're gonna have these uh these types of

guests as well and obviously we're gonna

have some entrepreneurs and uh business

owners that are already using these

types of strategies and tactics with or

within their own organizations and

that's why it's important in these first

episodes to

learn about all these concepts so that

if we

use these concepts or these words in

conversations we don't have to stop

because i mean the idea is to get as

much information as possible from these

these people this is this is the whole

idea of these first episodes let's let's

learn the basics so then we can have

very very deep conversations and we

don't have to stop

and and be like specific of what

everything means so today that's that's

the goal we're going to uh like

introduce you to a b testing what what

it means how it looks like and for this

i'm going to tell you a bit a little

story so

i started with e-commerce uh a while now

i at that time let's say 10 years ago i

was starting a brand you know cosmetics

a natural cosmetics brand and we already

had a small retail shop it was we were

doing very well we already we also had

some retailers that sold our brand we

had like minor retailers that but

basically bought store stuff from us and

then sold it in their own stores and or

their own businesses

and things were going really well but i

could see in other countries in other

more let's say advanced or developed

countries where i i was already seeing

how big brands were moving into

e-commerce and not only doing it in a

small scale but they were switching

their whole

way of doing business and and their

whole communication channels

essentially how they operated they were

switching all the way to e-commerce so i

said okay i think

even though

this wave has enriched my country

remember i am from mexico

so

i this wave didn't reach my country a

few years later but still i was like

okay i think i need to prepare myself so

i started at least i said okay

i want to start doing e-commerce so the

first thing i did is that i looked for

some developers right i at that time

that was the

smart the smart move okay so you you

find some developers that can help you

build the site your e-commerce site or a

store and that's what i did so

what what happened

it was a total a total fail i had to go

through four four different developers

to then realize that uh this was not the

way to go

and obviously i i don't want to put

place all the blame in these developers

the problem was at that time i i didn't

have experience about ux or ui i didn't

even know what it meant

and the developers didn't have any

experience in e-commerce or they don't

have experience in marketing you know

this was a time where not a lot of

people were selling online that the only

types of of you know sites that were

available at that time most of it was

you know these like click funnels

sort of things that you know the site or

the store as you can imagine it now it

wasn't that common before it it got

developed

through the years and it got optimized

through the years so at that time i

didn't have experience with ux ui and

obviously with development and the

developers you don't have a lot of

experience with marketing and and what

what should a site what is contained

what is a proper flow for a customer

there wasn't a lot of information

available at that time so that's why it

was a huge fail i at that time obviously

i

i did blame the developers now i i think

i am a little more mature as well but

now i have a better understanding of why

it actually failed so it failed because

we we don't know about like how to do it

in both sides but that

that experience led me to start finding

better solutions in this case i said

okay i'm an entrepreneur i want to make

this happen and i have failed previously

with somebody else doing the development

for me so i need to learn how to do it

there must be a way to

get you know

for a person that doesn't know how to

write code to create something that can

sell so i started doing some research

obviously at that time i learned about

you know wordpress uh and they were just

coming out with this uh with a new plug

plugin for

woocommerce

we already had magento but still it was

very very complicated for me i i don't

know anything about coding and and i

didn't know anything about hosting and a

lot of things i didn't have technical

technical background essentially i was a

merchant you know i i used to sell stuff

boy i didn't have a technical background

so magento was at that end for me and

then a friend recommended me shopify so

shopify

10 years ago it wasn't the platform that

we had today you know it's super robust

and you can do essentially anything with

shopify you can sell almost anything

with shopify at that time it wasn't as

robust as we have it today but still it

was a very very good option for someone

like me that wanted to start selling

online and they don't have the technical

background to to do it

so i

dug in i i took all the courses i did

all the you know like all the reading

and the articles and the blogs and

everything and i learned how to

basically create my first store

and i think that one of the reasons that

i succeeded at first at creating the

store was that there were templates so

before when when i was trying to do it

with developers there were no templates

so once once you get to this like blank

blank canvas and you say okay what

should we should we add to this to this

site you get froze you know you you

freeze and and then

what happens is that if you don't have

the experience or the technical

knowledge to actually create something

now the developer is going to ask me

okay why should i why should i put here

in in the homes in you know in the home

of of the site and i don't have the the

background as well of ux and ui so i'm

going to say i don't know you do it and

that's why it didn't work so with

shopify we start with a template and we

just have to fill in the blanks

to create something that is useful and

that it's already tested for you know

for its selling capabilities so

i ended up using shopify and and i could

create my own site and eventually

you know long story short eventually

this site became a very successful

business actually uh we at that time we

already have

maybe 15 retail shops and and 300

wholesalers

and the online store

could sell more i mean outsold

all of these channels combined and this

business eventually became a million

dollar business you know a million

dollar a year in sales business and it

was all thanks to to e-commerce you know

and and it still is this brand is still

very very successful because it's

e-commerce platform and and so that what

seemed very interesting here

is that we could we reached 1 million a

year in sales without even having to

know

a lot of technic technical stuff you

know or having a technical background

thanks to shopify now

you can say okay well yeah

what a great what a great story or what

a blessing but the problem is and and

this is what i want to address the

problem is that

since we never needed this technical uh

background

at least in i don't know maybe i i don't

need to know about coding but there's

certain stuff that you need to learn or

we need to learn along the way and one

of them is a b testing so when you grow

with uh with a platform like shopify

where pretty much everything is already

set up for you and you just need to

tweak certain things to make it work

sometimes you you don't grow with this

culture of experimentation okay what if

we draw a little bit out of the lines

and and maybe we run some tests

and and

have like two examples with with shopify

this doesn't happen organically and then

the problem is that when you reach to a

certain level

uh let's say when you have a million

dollar business and use and you want to

start investing a lot more money in

marketing

or or you you want to just bring more

traffic into your site

but obviously you want this traffic to

convert because it's not it's gonna be

like way more expensive for you so you

don't know how to test and because you

didn't need to learn that in the process

so

i learned uh very late let's say late i

i know i'm i'm a little hard on myself

i think all entrepreneurs are

that's we we criticize ourselves a lot

because we are perfectionists and we

always ask ourselves how can we make it

better so i like i for me was i learned

a little bit late that there's stuff

that you can do

like a b testing so your site can

perform and even outperform other sites

or your your competition with less money

and this is what a b testing actually is

or it allows you to do so now to define

this concept what actually is a b

testing it's very very easy to

understand a b testing

is

basically running an experiment

but with tools that you can

insert in your site that will allow you

allow you to measure

these experiments okay it's not okay

let's just let's just do something let's

just do two versions or of something and

see which one is better it's not it's

not as simple as that it's also not

super complicated you will need certain

tools to have accurate and correct

measurements of your experiments but

essentially is having two versions of

something and we're gonna give some

examples but having two versions of

something and see which one performs

better so let's say that we have a site

like shopify okay or or our site is is

powered by shopify we we still have a

lot of elements

in our home page that we can test with

with with other other versions of the

site for example

did you know

that more than half of the people that

visit your site in their first session

won't

see other places of your site but the

home page and specifically

right where the banner is they call they

call this the fold so the fold is this

imaginary line from where you first land

okay you you land in the home page and

you can see the banner and the menu and

there's an email imaginary line right at

the at the bottom where if you scroll

if you you do a scroll

what's before

what's below that line is uh it's below

the fold so that in my imaginary line is

called the fold so more than half of the

people

that land in your sight for the first

time won't go beneath the fold

and won't even click on anything like on

the menu or on the banner so this is

like uh general statistics so

this first page or first impression

is very very important for your site and

and here we have a lot of elements that

we can actually test so for example we

have the banner so in your banner you

have a lot of

assets so you have a background color

you have uh possibly a text you know

like a headline a big headline you also

have a button with a call to action

maybe like right below the banner you

have like a category product or your top

sellers

but you can still see some of it above

the default you have a menu as well you

have uh you know that this tiny banner

that is above the menu so there's quite

a lot of elements that that are in that

first page or in that fold

that you can actually test

so let's take the headline just as an

example imagine that you have

you right now you already have the

version of your site that is published

and and you already have a banner design

and you already have a headline for that

binary design so if you want to do a b

testing what you need to do is

first of all get a plugin like convert

so you can measure the results and then

you can create another version

of this headline another text and run it

so what's going to happen is that this

the softwares will send

will split the traffic into in these two

versions of the sites

we

when the first one that we call it

control

the first half would go to the control

site with the normal text or the normal

headline that you started with and half

of the traffic is going to go to the new

version of this headline and at the end

you'll know

which one worked better okay now at the

end you have an

an answer okay so

the question was okay will it work

better

if i write these words versus these

words or this copy at the end of this

experiment you'll have an answer and the

answer is yeah it actually worked better

so now you have an experiment obviously

it's a little bit more complicated than

that because there's certain parameters

but we're gonna get to those

further further along the way

in the next episodes right now we just

want to understand the the actual

concept of a b testing so

you can also do this with the

the color

the the picture or the images that you

have in in your actual banner you can do

also with the color of the button and

the text that you have in the button of

your call to action so you can

a test

by now or more information or buy now

with see all the products or i don't

know you you can run a lot of a lot of

tests so the real goal of a b testing is

to switch from

you know the the what we all

all entrepreneurs do a lot and and this

is

i think

i believe for we want to switch those

words to i know okay i know because if

we know that something works better

than something else okay now we have

certainty that we can invest in bringing

more traffic because we already know

that that works better

now

if we already have an answer

we run an experiment and we have a an

answer between two tests so now we know

that the test number two or the version

number two works better we can invest

more money or we can also do another

test okay now let's see if this one

version three

works better than version two and we can

just keep dialing in and dialing in

and eventually have the just the perfect

the perfect copy i mean obviously we're

not gonna get to perfection but almost

perfect you know what what's the almost

perfect pitch or copy or headline or

button or caller or whatever

that converts the most and this a b

testing is

the quintessential uh way to do

basically experimentation and

optimization for our sites like i said

we are going to get really more into

details and we're going to dive deeper

into how to do it in other episodes

right now i just wanted to you know give

you a

a little bit of understanding of how

this works so to sum it up a b testing

is just running experiments on your site

having one version with which is your

control version and running against

another version of the text or the

caller or the picture or the offer or

the price there's a lot of ways to do it

and see which one works best

so then we can drive more traffic

knowing

that this is going to have better

results for our site and obviously that

means for our sales so i hope i hope

this helps i hope this information will

uh at least

get you excited about you know getting

getting uh a b testing in in your

ecosystem in your organization this is a

b testing and every topic that we're

going to be talking

in this podcast

it's a lot about culture you know having

this culture of experimentation and data

within organizations this is the most

important thing

all all of the other aspects you know

the technical aspects and the just the

the small details we're going to cover

those and there's already softwares and

agencies and a whole a whole ecosystem

that can help you with that but you need

to have that culture first in your

organization and that's one of the goals

also of this this podcast so again thank

you very much if you have any questions

ideas topics that you want us to cover

please don't hesitate to write to us uh

and obviously please subscribe we're

just getting started we're gonna have

some great uh great talks a little

further down the way and some good

guests and also if you know someone that

that loves these these topics about a b

testing e-commerce also you can refer

them to us and we'll have a

discussion here in the podcast so thank

you very much and see you in the next

one bye

[Music]