How to Get Users and Grow - Alex Schultz, VP of Growth at Facebook - Stanford CS183F: Startup School

Summary

In this insightful talk, Alex Schultz, Facebook's VP of Growth, shares key strategies for driving user growth and retention. He emphasizes the critical role of product retention, illustrating how a strong retention curve can predict a product's viability and lifetime value of customers. Schultz discusses the importance of identifying 'magic moments' in a product's lifecycle that enhance user engagement. He also covers various tactics for user acquisition, including SEO, social media, PR, and paid advertising, while stressing the necessity of product-market fit as a foundation for growth hacking. The speaker advocates for a data-driven approach to understand user behavior and iteratively improve growth strategies, highlighting the significance of aligning a company around a clear growth goal.

Takeaways

• 📈 Retention is the most critical factor for growth, as it indicates how engaged users are with the product and their likelihood to return monthly.
• 📊 The importance of calculating and analyzing retention curves cannot be overstated; they provide insight into user behavior and the potential for long-term success.
• 🔍 Data is essential for understanding user behavior, predicting future trends, and making informed decisions that can accelerate growth.
• 🌐 Network effects, opening new interfaces, and adding categories can significantly impact growth by increasing user retention and attracting new users.
• 🎯 Identifying and delivering 'magic moments'—the experiences that delight users and make them loyal to the product—is key to driving retention.
• 💡 Personalization in marketing, such as using social context and profile pictures, can significantly enhance user engagement and conversion rates.
• 🔑 Having a clear, company-wide goal, such as monthly active users (MAU), aligns the entire organization and prevents conflicting efforts.
• 🚀 Growth tactics are most effective when there is an existing 'spark' of product-market fit and retention; without these, tactics are unlikely to succeed.
• 🛠️ A/B testing is a vital tool for predicting what will happen when features or changes are rolled out to all users, and should be trusted as a decision-making instrument.
• 📝 Logging and understanding every critical flow in a product can be crucial for diagnosing issues and identifying growth opportunities.
• 💬 Communication, particularly through the correct channels like email, SMS, and push notifications, needs to be optimized for the marginal user, not just power users.

Q & A

What is the most important factor for growth according to Alex Schultz?

The most important factor for growth, as emphasized by Alex Schultz, is retention. He explains that having a product with great retention is crucial for driving growth, as it ensures that users return to the product at least once a month.

Why does Alex emphasize the importance of monthly active users (MAUs) over daily active users (DAUs)?

Alex emphasizes the importance of MAUs over DAUs because MAUs indicate that users are engaged enough to return to the product at least once a month. This metric is more reflective of long-term user engagement and retention, which are key for sustainable growth.

What is the significance of a retention curve and what does Alex suggest looking for in it?

A retention curve is significant as it shows the percentage of users who remain active over time since their acquisition. Alex suggests looking for a curve that flatlines above zero, indicating that a portion of users consistently return to the product, which is a sign of good retention.

How does Alex define 'Magic Moments' in the context of product growth?

'Magic Moments' are defined by Alex as the specific instances when users realize the value of a product or service. These are the moments that make users say 'yes, I've got it' and are pivotal in driving retention and satisfaction.

What is the role of network effects in driving growth, as explained by Alex?

Network effects play a crucial role in driving growth by increasing the value of a product or service as more people use it. For instance, when friends join a social network, it makes the network more attractive for existing users, thus increasing their likelihood to stay and engage with the platform.

How does Alex suggest using data to understand and improve user retention?

Alex suggests using data to track user behavior and identify patterns that correlate with retention. By analyzing metrics such as the number of friends a user has on Facebook or the first purchase on eBay, companies can find 'Magic Moments' and optimize the user journey to hit these moments faster.

What is the concept of 'growth accounting' that Alex introduces?

Growth accounting is a method of looking at growth by considering not just the number of new users acquired, but also the churn and resurrection rates. It involves understanding the net growth as a result of new users, churned users, and users who return to the service.

Why is having a clear goal for the growth team important according to Alex?

Having a clear goal is important because it aligns the entire company towards a common objective. Alex explains that when everyone is working towards the same goal, such as increasing monthly active users, it ensures that all efforts are coordinated and directed effectively.

What does Alex mean by saying 'data gives you empathy for your users'?

When Alex says 'data gives you empathy for your users', he means that by analyzing user behavior and feedback through data, teams can better understand the user experience from the user's perspective. This understanding can lead to more user-centric design and decision-making.

How does Alex view the role of a growth team in a startup?

Alex believes that in a small startup or even a Series A company, the entire company should be focused on growth rather than having a separate growth team. The purpose of a startup is to grow, and thus everyone should be aligned and working towards growing the key metrics that define the company's success.

What is the importance of 'Magic Moments' in driving user retention and satisfaction?

Magic Moments are pivotal in driving user retention and satisfaction as they are the defining moments when users realize the value of a product or service. Identifying and optimizing the user journey to reach these moments faster can significantly enhance user engagement and retention.

Outlines

00:00

📈 Introduction and Growth Fundamentals

The speaker introduces Alex Schultz, VP of growth at Facebook, emphasizing his expertise and contribution to Facebook's growth from under 100 million to 1.6 billion users. Alex discusses the importance of retention as the key to growth, highlighting the significance of monthly active users over daily or weekly active users. He refutes the notion that large user bases are needed to calculate retention, advocating for the analysis of user cohorts from day one of registration. The talk sets the stage for further exploration of growth tactics and metrics.

05:02

🔍 Understanding Retention Curves and Their Impact

Alex explains how to analyze retention curves, which show the percentage of users remaining active over time since acquisition. He describes different types of retention curves observed in various businesses, such as games with low retention, services like Minecraft with stable retention, and marketplaces that can see resurgence due to network effects or new platform availability. The goal is to achieve a retention curve that asymptotes to a line parallel to the x-axis, indicating a sustainable level of retained users. Alex also discusses the importance of 'magic moments' in user experiences that drive retention.

10:05

📊 Growth Accounting and Magic Moments

This section delves into growth accounting, which involves understanding not just user acquisition, but also churn and resurrection—users who leave and then return to the service. Alex emphasizes the importance of focusing on reducing churn and encouraging user resurrection to drive net growth. He also discusses the concept of 'magic moments,' those pivotal experiences in a product's use that significantly contribute to user retention and satisfaction.

15:05

🧠 The Role of Dimensional Reasoning in Growth Strategy

Alex introduces the concept of dimensional reasoning, using the example of British physicist Geoffrey Taylor to illustrate how understanding the units of measurement can lead to insights about a product's growth potential. He argues that the level of retention needed for a viable business varies by category and that it's crucial to determine the appropriate retention rate for one's specific market and product type.

20:08

📉 Analyzing Advertiser Retention and Revenue Curves

The speaker presents a case study of Facebook's advertiser retention and revenue curves from 2007 to 2008, demonstrating how these curves can predict future engagement and revenue. He explains the methodology behind calculating the revenue per user and how it can be used to project the lifetime value of customers. The example serves to show that even with a small dataset, meaningful predictions about retention and value can be made.

25:11

🎯 Aligning on Clear Growth Goals

Alex stresses the importance of having a clear, unified goal for growth, using Facebook's focus on monthly active users as an example. He discusses the challenges of aligning a team or company without a single, clear objective and argues against the notion that a dedicated growth team is necessary, especially for smaller companies. Instead, he advocates for the entire company to focus on growth, with each department contributing towards the same goal.

30:13

🔬 The Growth Planning Cycle: Understanding, Identifying, and Executing

This section outlines a growth planning cycle that involves understanding current metrics and user feedback, identifying opportunities for improvement, and executing strategies to address those gaps. Alex emphasizes the importance of logging data to avoid 'data debt' and the role of analytics in driving informed decision-making within a company.

35:14

📝 The Power of Data in Driving Growth and Empathy for Users

Alex discusses the critical role of data in understanding user experience, predicting future trends, and making faster decisions. He refutes the idea that there's a trade-off between user experience and metrics, arguing that proper metrics can enhance empathy for users. He also highlights the importance of personalizing experiences based on data to better serve a diverse user base.

40:16

🚀 Growth Tactics and the Importance of Product-Market Fit

The speaker explores various tactics for driving growth, including leveraging the first 100 users, using research and outreach, harnessing the power of social media and PR, and utilizing ads effectively. He emphasizes that while these tactics are important, they are only effective when there is an underlying product-market fit and retention, comparing the relationship to pouring fuel on a fire with a glowing ember.

45:17

🛍️ On-Site Merchandising and Personalization Strategies

Alex discusses the importance of on-site merchandising, such as personalized suggestions and creative strategies that leverage owned media. He highlights the effectiveness of personalization, social context, and the use of profile pictures to create highly relevant and engaging user experiences that can drive growth.

🎯 Targeting, Creative, and Conversion in Growth Tactics

The speaker explains the importance of behavioral targeting over demographic targeting, emphasizing that understanding user behavior is more effective for growth. He also discusses the role of creative content in growth strategies, arguing that while it is important, it is less critical than targeting and channel selection. Alex provides examples of how small changes in call to action can significantly impact acquisition rates.

📲 The Role of Social Media and the Importance of Context

Alex talks about the significance of social media in growth strategies, focusing on the importance of creating content that is interesting and shareable. He also discusses the value of custom audiences and look-alike targeting on platforms like Facebook for finding new customers similar to the best existing ones.

🚀 Embracing the 'Move Fast' Philosophy and Focusing on Key Metrics

In the concluding section, Alex reiterates the importance of moving fast, making decisions with 70% of the data rather than waiting for perfection, and focusing on key metrics for growth. He shares an example of how eBay shifted its landing pages to improve activation and revenue, illustrating the principle of prioritizing actions that lead to the 'magic moment' for users.

Keywords

💡Growth

Growth in this video refers to the expansion of a company's user base or customer base over time. It is the central theme of the talk, with Alex Schultz emphasizing the importance of growth hacking and strategies that can be employed to increase a company's size and reach. The concept is illustrated through various tactics and examples, such as the significance of product market fit and the role of retention in driving growth.

💡Retention

Retention is defined as the ability of a product or service to continue to engage and maintain its users over time. In the context of the video, retention is highlighted as the most critical factor for growth. The speaker discusses how a company can measure retention through cohort analysis and the importance of a 'flat line' in the retention curve, indicating that a certain percentage of users continue to engage with the product regularly.

💡Monthly Active Users (MAU)

Monthly Active Users (MAU) is a metric used to measure the number of unique users who have interacted with a product or service within a given month. The video emphasizes the importance of MAU as a key performance indicator for growth. Alex Schultz explains that focusing on MAU aligns the entire company towards a common goal and serves as a foundation for further engagement and revenue generation.

💡Magic Moment

The 'Magic Moment' is a concept introduced in the video to describe the instant when a user realizes the true value of a product or service. It is the pivotal experience that leads to user retention and satisfaction. For instance, on Facebook, the magic moment might be reconnecting with an old friend, which encapsulates the core value of the social network and encourages continued use.

💡Network Effects

Network effects refer to the phenomenon where the value of a product or service increases as more people use it. In the video, network effects are cited as a significant driver of growth and retention. The speaker uses examples such as social media platforms, where having more friends or users on the platform increases its attractiveness and utility for existing and potential users.

💡Metrics

Metrics in this context are the quantitative measures used to track and assess the performance of a product or service. The video underscores the importance of metrics like retention rates, MAU, and revenue per user. Adam D'Angelo's lecture is recommended for insights on metrics, and the speaker discusses how metrics can help in understanding user behavior and predicting future growth.

💡Product Market Fit

Product market fit is a term used to describe a situation where a product satisfies a market need and is well-received by its target audience. The video mentions that having a great product with product market fit is essential before focusing on growth tactics. Without product market fit, efforts to grow the user base may be less effective.

💡Growth Hacking

Growth hacking is a term used to describe creative and innovative marketing tactics aimed at increasing a product's visibility and user base rapidly. In the video, the concept is discussed with a focus on the importance of having a 'spark' or product market fit before growth hacking can be effective. The speaker advises that growth tactics should be used to fuel an already promising product.

💡Data-Driven Decision Making

Data-driven decision making is the process of using relevant data and statistical analysis to make informed business decisions. The video emphasizes the importance of data in understanding user behavior, predicting future trends, and making decisions that will foster growth. Alex Schultz shares his experience with analytics at Facebook and how it has been instrumental in driving growth.

💡Incrementality

Incrementality in the context of the video refers to the additional or extra value generated from marketing efforts beyond what would have been achieved without those efforts. The speaker discusses the importance of analyzing incrementality when considering marketing channels like pay-per-click ads, ensuring that marketing spend leads to new user acquisition rather than just existing users.

💡Custom Audiences and Look-Alike Targeting

Custom audiences and look-alike targeting are advertising strategies used on platforms like Facebook to reach specific groups of people based on their characteristics or behaviors. The video mentions these tactics as highly effective for startups to find new customers similar to their best existing ones, thus aiding in精准 growth and user acquisition.

💡On-Site Merchandising

On-site merchandising refers to the strategies used on a company's own website to promote products or services to its visitors. In the video, the speaker discusses the importance of leveraging owned media through on-site merchandising, such as personalized creative and product recommendations, to enhance user engagement and drive growth.

💡Behavioral Targeting

Behavioral targeting is a marketing strategy that involves targeting users based on their behavior or actions rather than demographic information. The video explains that understanding user behavior is more important for effective targeting than traditional demographics, as it provides a clearer signal of what content or products a user is likely to engage with.

💡Conversion

Conversion refers to the process of turning a potential customer into an actual one, often through actions like making a purchase or signing up for a service. The video discusses the importance of considering all other growth factors before focusing on conversion, and how elements like landing pages and call to actions significantly impact conversion rates.

💡Move Fast

The concept of 'move fast' is a core value at Facebook and is emphasized in the video as a key principle for growth. It suggests that companies should act quickly to execute plans and seize opportunities, even if they are not perfect. The speaker shares examples of how Facebook has embraced this value to make bold decisions that have contributed to its rapid growth.

Highlights

Alex Schultz, VP of growth at Facebook, discusses the critical role retention plays in growth, emphasizing that it is the most important factor.

Facebook's growth strategy involved focusing on monthly active users (MAU) as a key metric, aligning the entire company around this goal.

Retention curves are vital for understanding user engagement over time, with the ideal curve flattening out above zero, indicating ongoing engagement.

Network effects, opening new interfaces, and adding categories are key drivers that can cause retention curves to re-accelerate after initially declining.

Growth tactics are most effective when there is already a 'spark' of product-market fit and retention in place.

The importance of understanding and leveraging 'magic moments' in a product to drive user retention and satisfaction.

Data is crucial for empathizing with users, predicting the future through patterns, making decisions faster, and avoiding data debt.

Facebook's growth team uses a three-step process of understanding, identifying opportunities, and executing to drive growth.

The significance of dimensional reasoning in determining the necessary retention levels for a viable business, drawing parallels to historical examples.

The importance of having a clear goal for the company and aligning the entire team around that single 'north star' metric.

The value of owned media in on-site merchandising and how it can be leveraged for growth through personalized user experiences.

The role of targeting, creative, and conversion in effective advertising, with a focus on behavioral targeting being more important than demographics.

The impact of personalized creative in advertising, such as using social context and profile pictures to enhance relevance and engagement.

The importance of moving fast and being bold in executing growth strategies, as exemplified by Facebook's core values and past decisions.

Schultz's advice on tactics for acquiring the first 100 users, emphasizing the value of honest feedback and targeted outreach.

The significance of optimizing for marginal users in email, SMS, and push notifications to ensure that the right users receive the right messages.

Transcripts

Good afternoon everybody.

Today, we have Alex Schultz, VP of growth at Facebook, and

probably the world expert on growth, as he can vouch from this weekend,

the person I go to when I have a really hard growth question.

Many people who work at Facebook, many of these founders at Facebook in fact, and

very early employees have said hiring Alex has been one of the secret

successes of Facebook that is not well understood.

You joined almost ten years ago, and

I think at that point Facebook had less than 100 million users?

>> Yeah. >> Way less than 100 million users.

Facebook now has 1.6 billion and Alex has been after the importance

of having a great product, maybe the second most important driver of that.

>> Having Naomi, these very important people.

>> And the team, anyway, thank you very much for coming to speak to us.

>> Thank you.

I did not pay Sam for that introduction and I would note that Havi is my boss so,

hi Havi.

But no, Havi and Naomi, Danny,

they are these people who've been in the growth team for four years and

then people like Blake, and Martin, and Ray and James Wang who set the team up.

And it's very nice, but I am a VP on growth, I am not the VP of growth.

So you've had an amazing lecture series.

I just watched all of them yesterday to make sure I didn't say anything without

actually having seen what everybody else said.

And it was totally awesome.

I loved what D'Angelo had to say, so that lecture, if you weren't here, if you

didn't watch it, I strongly recommend you go back and see what Adam D'Angelo said,

because I could not emphasize his points on metrics more than enough.

And so what I'm going to try and do for

the first half is go over some of the points that have been made already and

emphasize what I think is most important when you're thinking about growth and

what my take is and our take is on the Facebook growth team on those.

And then in the second half, try and talk to you a little bit more about

tactics that have been particularly useful.

The caveat that I think you should take onboard is what Stewart from Slack and

Flickr said.

Which is to some extent, you do have 100 million monkeys bashing at typewriters,

and I am very, very lucky to be on the growth team at Facebook

to get to work on a product like Facebook that has the retention that Facebook does.

That has everyone in the world wanting to use the product and to say that,

I drove it or any of us drove it beyond their being great product market fit

is perhaps not true.

I can't tell you what the AB test is if we didn't even have a growth

team at Facebook.

So what matters most to growth?

The single most important thing to growth is retention.

Adam commented on this in his talk, and he showed you cohorts from some real company.

But here is a stylized cohort of how you should look at retention.

The number of days from acquisition is on the x axis.

On the y axis is the percent of people who has still monthly active.

I care a lot about using monthly active, not daily active, not weekly active.

There are really good arguments for using daily active that people say all the time.

Because it's about intensity of usage.

But what really, really matters is that you got skin in the game and

people come back to your product at least once a month.

And then you build on top of that engagement revenue,

whatever your key metrics are which we'll get to in a minute, but firstly,

I believe monthly active is the most important thing.

The second thing is how do you rally calculate it?

So I've got a real graph for revenue in a few slides from Facebook from

the first 250 days of our ads products, which I'll show you.

But the thing that's really, really important is to say that,

it's very easy to calculate this curve even if you have 1,000 or 100 users.

And what you do is you look at all of your users on the first day

after they signed up.

Were they monthly active?

Yes, clearly, right?

The first 30 days everyone's going to be monthly active.

I'd prefer if you defined a month as 28 days, by the way, because multiples of

seven is really, really good for these kind of metrics because of weeks.

But for the first 30 days, everyone has been active on their first day,

on their second day, on their third day as a monthly active user,

whether they came back or not.

And then what you do is you look at day 31.

And obviously, only some of your users will have been active for 31 days.

Everyone who signed up a day ago was not active for 31 days.

So what percentage of them were active on day 31,

what percentage of them were active on day 32?

And as you go further and further out in the number of days since registration,

the number of people who have been on your service that long declines.

Which means that your metric is going to get noisier, right,

as you get out to the far right hand side of that curve.

But that doesn't mean you can't get a handle of where things flatten out very,

very early on with very small numbers of users.

The number one criticism people give me when I talk about retention as a metric.

Or even when you look at what Adam presented in his presentation earlier is

you need lots and lots of users.

So you need millions of users to calculate that, that is not true.

We did it with thousands of users in the early days of advertisers for Facebook for

example.

Or you need to have had all of those users on for a very long time.

Look at Adam's graph it was multiple years with months, and months, and months.

No, you can do it day by day, and

you can get to a really good place super, super fast, using this methodology.

So what you're looking for in a retention curve?

You're looking for it to flat line above zero, not flat line like a heart monitor.

You want it to flat line at some line that asymptotes with the x-axis.

There are three types of these curves that I've seen.

The first type is where asymptotes to the x-axis.

Like really, really close, goes down to almost zero.

You hear about a lot of those companies.

You think about most games, most games are smash hits.

How many people use FarmVille now versus how many people used FarmVille at its

absolute, please tell me you know what FarmVille is.

[LAUGH] Yeah, okay, all right.

How many people used farm, I'm feeling old.

I had hair when I first did one of these.

How many people use FarmVille now versus how many people used to use it?

FarmVille was a smash hit, then there's other ones which look like this,

where they stabilize out.

So you think about Minecraft, right, or World of Warcraft.

These games that somehow have network effects and people stay in them for

a long time.

The other version for this curve I've seen is the ones where they go down, they look

like they're going to hit the x-axis and then they re-accelerate and they go up.

There's a few different things that can cause that right, so you have this,

this overwhelming thing that is driving you down to have zero retention and

somehow the numbers go up again.

So things that drive that for businesses, and

you actually I think mentioned this in your intro.

One, network effects, so

when all your friends get on Facebook, you're much more likely to use Facebook.

When all your friends get on Twitter,

you'll more likely use the Twitter, WhatsApp, Messenger.

Whichever social service you're talking about,

there are network effects that really matter.

Network effects also count in marketplaces.

When you have enough sellers, you can have enough buyers.

When you have enough traffic, you get more sellers.

And so as marketplaces go, often what you find is these old cohorts that signed up

resurrect and come back to the service.

So network effects can drive the curve up after it's flatlined somewhere or

is heading towards zero.

The second thing that can do this is opening up a new interface.

So think about it.

A lot of people release their apps first on iOS.

I actually loved Diane's talk last week for this, by the point that he made of

starting with Nokia, focusing on Android because it's what most of the world use,

but in Silicon Valley we typically build our apps first on iOS more often than not.

And then when someone releases an Android application, and

this was something I saw with Instagram years and years ago.

But when you release an Android application,

people might have tried it on an iPhone And then traded their iPhone for

an Android and no longer be able to access your service.

People might've tried your service on a friend's iPhone and

then when you add an Android they come back and use your service.

So that's another thing that can drive you up again.

And then the third thing that I've seen on this curves that drive this curves up

again is adding categories, so think about Amazon.

There was a really good discussion, was that Stuart earlier or?

Someone earlier in this talked about Amazon, and how Amazon started with books

because there was this large inventory available of books.

But then they added in another category.

And then they added in another category over time.

And imagine, there will be people who came to Amazon to buy that one book, but then

two years down the line Amazon suddenly had kitchenware, I don't know, or fashion.

And they came back and then they bought fashion and

they brought them more use cases to come back more monthly active.

This was true with eBay.

We added motors.

I remember being at eBay and we had to do all of our reporting on

gross merchandise volume with and without motors.

And no one had ever believed that you could buy a car online, and eBay did it.

And it created a whole new use case for people, many of whom were existing eBay

customers, who now had one more category they could shop in.

So you want to curve the asymptotes to a line parallel to the x-axis.

You can do it with far more limited amounts of data

than most people think you can by using the technique I outlined earlier, and

I'll show you a real world example in a few slides.

And there are three types of curves that I've seen.

Curves that asymptote like this, awesome, your company will survive.

And how big will it get?

We have another question.

Curves to go straight to zero.

That's okay, you could be in games and

your job is to produce one hit after another after another.

EA doesn't expect whatever fee for 2012,

they still have lots of users now if they've released FIFA 27.

So it's okay, there are businesses where hitting the x axis is okay.

But you need to know you're in one of those businesses.

And then the third that I've seen is the curve going up.

And that can be driven by network effects.

That can be driven by being available on more platforms.

That can be driven by adding categories to your marketplace or service.

From this falls out a brilliant way to look at growth, and

this guy Danny Ferrante who was at Yahoo.

Who was one of the first members of the growth team runs core data science at

Facebook, he's totally awesome.

He's really the brains behind the operation I'm the mouth.

Danny came up or brought this to Facebook, and it's a simple way to look at growth.

We were looking at growth as how many users were we growing by week on week?

We looked at it on an absolute basis.

And looking at it like that we were like okay so

we're signing up whatever it was a million a week and

we're growing by whatever it was 800,000 a week I don't know.

Cool, we're churning 200,000 people.

And we never actually looked at it in this way.

When we change the way we look at things to say well okay,

the net growth is 800,000 a week, the new users are 1 million a week.

But what is churn and resurrection?

And what we actually found at the time, and

I can't remember the absolute numbers so I'll just make some up.

But imagine you're churning 1.3 million.

So not 200,000, 1.3 million.

You're resurrecting 1.1 million.

So plus 1 minus 1.3 plus 1.1,

you've got net 800,000 in growth.

But that 800,000 in growth is being driven,

I think I did the math wrong, but 800,000 in growth is being driven by

a different order of magnitude from churn and resurrection than you realized.

And so that helps you focus on how do I stop people churning?

How do I get people to come back to my service who are not coming back to my

service right now?

So once you have your retention curves, and

once you start to have a business with retention.

You need to look at your growth accounting,

not just how many users are we acquiring.

Make sense?

Overall the number one thing I focus on for growth,

and the one thing that we focus on for growth is retention.

So what gets you retained in any given service?

Magic Moment.

What is the moment when you used Airbnb?

What is the moment when used Door Dash, when you used Slack,

when you used Facebook, LinkedIn, WhatsApp?

Whatever the product, what is that moment when you went, yes, I've got it.

Right, on Facebook it's friends.

You see that first friend,

that person who I've got completely the wrong crowd for this but the person

from high school who you lost contact with as you grow older and moved away.

But you see that person and you go, wow.

I haven't talked to them in years.

Let's see what's going on in their life.

And they've had a kid and lost their hair too, and you feel better about yourself.

That is a magic moment on Facebook.

So how quickly can you drive the user to that magic moment.

You look at Airbnb, right?

They talked about the delight that they give people.

They talked about these different screenshots of the stages of the user

journey that they've created as a map of the user journey.

Getting that first person reserve your house.

Staying at that first property, those are our magic moments.

eBay, buying your first item from a stranger online.

Nowadays that doesn't seem crazy but back when I was working at eBay in the UK,

that was revolutionary.

And people really, their eyes lit up when you talk to them about it.

In Germany they had this great ad, tights by minds that showed the build up of when

you are waiting for the auction to finish and view to make your first purchase.

That was such an adrenaline endorphin moment.

But users loved it.

Nowadays, it seems a long time ago.

Those magic moments are what you need to look for.

So how do you find them?

Well you can find them in the non scalable way which is

don't they just sound relatively obvious?

You can actually think through what is the experience on my product?

What do I want people to do?

You can ask your first users, okay, you're keeping using my product.

Why are you keeping using my,

what is the thing that makes you happy to use my product?

What was that first moment?

It's non scalable but it generally works.

When you ask people what is the best thing about Facebook?

The first thing they say is connecting with friends and family.

In all the research, why do you like Facebook?

It helps me connect to my friends and family.

So you can do it the non-scalable way or you can do it the scalable way.

Look for correlations.

So Marcus talked at startup school himself and

he said that we focused on getting people to ten friends in 14 days.

It's a smooth curve, right?

Number of friends on the x axis and percent retained on the y axis.

It's not like there's a step change at ten, if you get nine you don't retain.

If you get 11 you do and it all changes at ten.

No, it's clearly a smooth curve and we picked a point on the curve.

But there's a strong correlation between

number of friends you've got in the first 14 days and whether you retain or not.

There's a correlation between number of friends in the first day but

there are correlations for any product like did you place a bid,

did you do a buy it now or did you list an item on eBay?

Was the thing that was completely correlated with retention for eBay.

You can imagine, I don't know, I mean DoorDash was founded at Stanford, right?

You could imagine for DoorDash.

You've placed your first order and that order arrives to your door.

And whether you place an order or not and how many orders you place you place in

the first week, probably they are actually correlated with whether you retain or not.

So you need to look for the magic moment of your product by talking to users and

then try and validate that with correlations.

Makes sense?

Cool, I love this.

I put it in every presentation I could.

So I'm lucky enough to have been trained in physics at Cambridge in the UK.

And my favorite and my favorite lecture series, not my favorite lecture but

my favorite lecture series was in this thing called dimensional reasoning.

And so it's easy for me to stand up here and say, okay,

you need to get flat retention and then you have a viable business.

That's not actually true that's kind of a glib statement.

If you're producing a social network and you have 5% retention you're

not going to long term have a tremendous social network.

Or a messaging app like what use would WhatsApp be if only 5% of

your friends actually used WhatsApp who signed up for it.

I mean it would be useless, right?

And there's no way we could release 1 billion plus numbers of active users if

that was true for us.

On the other hand, if you have a marketplace that is selling fashion.

And you have 5 or 10% retention but

those people spend a lot of money you have a totally viable business.

And so the level of retention you need varies dramatically

by the category that you're in.

Obviously, the higher the better is true in everything.

If you can have an Ebay type business model or

Amazon type business model with 90% retention.

You probably end up bigger than Amazon and that would be incredible.

But I think the point I like to make is, you can figure out and

I love this as an inspirational story.

So Geoffrey Taylor was this British physicist,

he was on the edge of the Manhattan Project,

he wasn't in the corp, he ended up winning a Noble Prize later for a different area.

And the US and

Russian governments were releasing these images to say this is how big our bomb is.

But they weren't releasing the actual power because that was a closely guarded

state secret.

And so what Geoffrey Taylor did he used this technique called dimensional

reasoning where you look at the units in the metric you were trying to achieve.

So joules, right, for energy, it's joules.

Kilograms times meters squared over seconds squared,

that's what joules decomposes into.

In this case, kilograms, what are you going to use?

The volume of the air times the density of the air, which handily, is one.

So you've now got meters cubed times meters squared over seconds squared.

So meters [INAUDIBLE] over seconds times seconds minus 2.

The time is on here, the diameter, the radius of the sphere is on here.

You just plug the radius in as m, you plug the diameter of the time in as

seconds, and you end up getting 25 kilotons as the output.

You get 10 to the 14, I think is what it works out as.

And it turns out the bomb was 21 kilotons as measured by the US government.

He could take one photo and

figure out the most closely guarded state secret in the United States at the time.

You can figure out what retention you need to have if he can figure that out.

>> [LAUGH] >> So

here is what I promised, this is an actual curve.

I presented a few different events, so

I taken off the numbers because I'm worried they're State secrets.

But this is from 2007 to 2008.

I joined the week we started our ads business at Facebook,

our self-service ad business by us.

The pink speaks to the right-hand axis, the blue speaks to the left-hand axis.

What the blue is is how many advertisers had been in the system at least one day,

two days,

three days because the x-axis is number of days from acquisition make sense.

So the thing has only been around 250 days, so

0 people have been advertisers 251 days ago.

And when they turned 51st date and all the people have been active on day 0.

And then each of those pink data points is the total revenue achieved from every user

who had registered.

Who was using the service on their 30th day, or

their 31st day divided by how many users signed up 30 or 31 days ago.

Not how many were active, right?

How many signed up.

So total in the cohort, who signed up 30 or 31 days ago.

You divide the revenue they made in the day by that number of users.

And that gives you that shape of curve that's up there.

I stumbled over my words a bit there.

I promise you if you look over it again it makes perfect sense.

I did this with Danny back in 2008.

The really important things I want you to take away from looking at this curve is,

look how flat the pink line is.

Like I've been talking about curves flattening out,

this is an actual curve flattening out.

And look how quickly it's flattened out.

By 100 days you can pretty much predict what the line is going to be at 200 days.

I've seen this over and over.

I've seen this with engagement.

So if you divide time spent by the same denominator.

I've seen it actually for sentiment.

If you have large enough sentiment survey and

you divide it by that same denominator.

I have seen it for posts.

I have seen it for photos uploaded, messages sent.

You can use the same techniques that I talked about for

determining how much retention you have and whether your business is viable,

to figure out what is the lifetime value of one of your customers.

If I was to acquire the total addressable market for

my business today, what revenue will I be making in 100 days?

That's what this curve a allows you to do.

You can predict it for posts.

You can predict it for page views.

Time spent, whatever you want.

The same methodology works for engagement, that works for retention.

And, actually, retention is often the biggest driver of this metric.

And we did this with very little data.

Later on, there's a stat we are okay releasing, which was end of 2012,

we had about 700,000 active advertisers.

Beginning we had about 300,000 active advertisers.

And this was five years before that.

So you can imagine there was a very small data set that was behind calculating this.

The last thing is I said things get noisy the further up you go.

You can literally see that on this graph, right?

The variance in the numbers the further to the right you go on the graph.

So now, how do you operate for growth?

Most important thing is have a clear goal.

Brian, who is awesome on growth marketing on Facebook,

great guy, he always says, if you are a growth team your product is a number.

One of the most brilliant things I think that Mark did in the early days of

Facebook was he aligned the entire company on monthly active people as our goal.

Now why does this matter?

It matters for a bunch of reasons, but the biggest and

most important reason is I think it aligns everyone.

When you have a company of more than two or three people, or by the way,

a team of more than two or three people, certainly when it gets to the hundreds.

You can't control what anyone does.

I mean, you could probably control what one person does, but

then the other 199 would do whatever the hell they wanted.

So what do you want to do?

You want to make sure everyone is pointing in the same direction, and

that direction is generally right.

So let's think back to the early days of any company.

Whether it's Facebook, whether it's Air BnB, LinkedIn,

whatever you want to look at.

It would be completely reasonable to have a goal of revenue, right?

You want your company to stay alive, you want to make enough money that your

company can still be viable, that you can employ people, that valuation goes up.

So a reasonable person could say, I think we need to drive towards revenue.

At the same time,

you could have a metric that was like, if you're Air BnB number of hosts.

If you're Facebook, number of posts.

If your DoorDash number of orders.

I don't even know what this company's metrics are beyond Facebook's.

You could easily have a production metric or a consumption metric.

You could have a time spent in our service metric, and

every one of those metrics would be a completely reasonable metric to drive for.

So you've got ten reasonable people,

maybe five of them think you should have monthly active people.

One is really focused on revenue, two are looking at time spent and

two are looking at number of posts.

Everyone is pushing in a reasonable direction that is vaguely aligned but

probably will have conflict.

If you want to get ad in front of users as quickly as you possibly can

you may not necessarily want to put friends in front of them immediately.

If you want to get them to post you may dial up the composer and

push down the ads.

I don't know. These are trade offs that you could

be making.

And if the whole team isn't clear what the goal is, they will be in conflict.

So having a clear goal that your company looks to drive is really,

really important, and

having that goal be MAP in my opinion is the best metric that you can drive for.

Because it is that once a month visitation,

that base level you can drive off of.

If you're going to do a consumer internet company.

But fundamentally, the most important thing is pick a goal,

your team's product is a goal, focus on that goal.

Align the company around that goal and

then you are going to operate correctly for growth.

The other thing I always say is you don't need a growth team.

If you're a small company, if you're even series A, don't have a growth team.

Don't hire someone like me.

The whole company's job should be growth.

I'm very happy at Facebook, so I don't want a job offer.

But the whole company's goal should be growing.

The purpose of a start up is to grow.

So, what is the number you want to grow?

And align the whole company against growing that number.

Does that make sense?

Now, Naomi is amazing and Naomi is someone people don't know that well

out of Facebook, but she runs product and engineering for growth.

She's brilliant, she's been at Facebook 12 years.

So, when you say I'm original, I feel not original everyday.

She sits like three feet from me, she's amazing.

Anyway, she has this approach that we use to iterate through our growth

planning cycles, which I think works really well.

Understand, first of all understand what is going on.

Look at all the data you have, talk to your users.

Ask your users what they think.

Understand everything that is going on with your product that you can

possibly understand.

What matters?

Retention.

Are you churning too many?

Do you have the retention?

Is the revenue going up on a per user basis?

What are the acquisition channels you're getting?

Which channel are people resurrecting through?

Are they resurrecting through email?

Is it push notifications?

What's going on?

Understand what's going on for your system.

Have that all tracked.

Then identify, look through all of those different metrics, all of that different

research you get in, and identify where there are opportunities.

Where do you see big drop-offs?

Do you have a lot of people with push notifications turned on

who are not receiving push notifications?

Do you have a lot of people who are receiving push notifications who

are not clicking on them?

Do you have loads and loads of social media traffic, but no search traffic?

Are your email viral invites working but your SMS ones not, and

what is the difference between those?

Go and look at the data and identify where there are gaps, and then try and

double down on fixing those gaps in the execution phase.

Now to this day, this is still how we run growth at Facebook and

you can even see a a talk Naomi and I gave online nine years ago I think at FA.

Eight or nine years ago, where we talked about using this approach and

we have kept it going.

Understand what's going on, identify the opportunities in that data you've

understood, and then execute against it.

Another really interesting thing I think was said earlier is data debt sucks and

you're going to regret it later.

And your intuition is going to run out at some point and

then you're going to want data.

In 2009,

we actually shut down all of the work the growth team was doing during January.

And we only spent the time logging, and

making sure every critical flow on Facebook.

Registration, at the time we didn't log clicks on emails.

So, click some emails, it seems obvious now but we didn't have it.

Clicks on the emails,

did you then actually log in or did you fail in the password entry.

All of these different things, we logged them so

that we could understand what was going on.

And that was super critical because actually in March that year,

we had a massive growth slow down.

And we would not have been able to understand what was going on if we hadn't

done the logging in January.

Because what went on was actually super new ones and

without that logging in place, we wouldn't have known.

Get the data logged so you can run through this process.

So speaking of data, I am very, very lucky at Facebook.

I do growth marketing, I've run the awesome internationalization team.

But I also run the analytics team for the company and

that's been a couple years now.

When I talk about data to people internally,

I like to say these three points.

There's a meme in the valley that you operate either for user experience or for

metrics.

And I really want to bust that meme.

Data when used in the right way gives you empathy about your users.

One example I've talked about in the past is we launched a product on

Facebook that you may have seen on your big 42 inch monitors.

No one has a big 42 inch monitor in here, but I'm sure some of you do at home.

That has on the right-hand side all the friends you can chat with on Facebook, but

at the top it has this list of stories that's ticking over called ticker.

When you're on a small screen,

that product used to collapse into the right-hand column of Facebook and

push things down so that you could still see these stories ticking over.

That product worked great for all of us who had

huge numbers of friends who are creating a lot of content on big screens.

But if you went to the Philippines where there was huge usage in Internet cafes and

people run narrow screens,

the majority that amended up having that in the right-hand column.

And the majority of them were low engagement with Facebook and

couldn't see people you may know below the fold.

And so, we actually lost friending amongst users who are very different to ourselves.

And we only figured out what was happening by looking at the data and

saying, why is this bad on small screens and awesome on big screens?

Why is it not so great for low engagement users and super for power users?

It was because we used the data to understand these users who are different

to us here in Silicon Valley.

Now think about it, I presume I don't know.

I'm seeing a lot of Apple products here.

I have a Lenovo at the moment.

But most of you probably have a iPhones.

Who has a iPhone?

Who has an Android?

You're the best.

Most people in the world are using Androids.

But everyone in this room has an iPhone, first and foremost.

You don't understand.

I'm actually forgetting how to use iOS now,

which actually is a problem in my job.

But I've got an Android because the majority, maybe not the majority,

a vast number of our users use Android, as Jan said in the last talk.

And I need to understand what the Android operating system offers.

But I can't understand Android and iOS well as a user at the same time.

So, I used data to understand what's going on on iOS.

You need to gain empathy for your users.

And research is great, and talking to your customers is great.

But if you listen to the Twitch guy,

Emmit, talk to your users then go away and build the product to serve them.

And look at the data, in my opinion,

would be the next thing to validate if you built the right thing.

If you're doing the right thing,

if your research actually ties up with what your users have said to you.

So, data gives you empathy.

It's not a trade-off between metrics and users.

If you build the right metrics, you are optimizing for the users.

Two, data predicts the future.

Over and over and over again, I love this quote.

It's a science fiction outright thing.

But like the future is here, it's just not evenly distributed yet.

So, those curves that show you the retention.

When I said you can look at your total addressable market and

say, if I sign them all up, how much revenue would I get?

Or you just look at who you signed up today.

Let's say back in those days, we signed up a thousand advertisers in a day.

I could tell you how much revenue they would contribute

hundred days out because of that curve.

Now, also you're users are going to ramp up.

So, a user who shops in one category if you're eBay or if you're Amazon and

then buys in a second category and

a third category is clearly going to get more valuable, right?

Shopping the shop, driving up same store sales is a common technique in e-commerce.

Okay, so

how many of your users are increasing the number of categories they're shopping in?

You see someone sign up.

What likelihood is it that they're going to add extra categories, and

what is your revenue going to be in the future?

You can predict that with data.

AB tests are another great thing.

See all these people walking in the science march believing in science.

And then when you apply the scientific method to their product and

run an AB test, they ship the thing that looked bad in the AB test

because they didn't believe the AB test.

I see that with all kinds of startups I talk to.

AB tests tell you what's going to happen when you ship them to 100%,

use them it works.

Finally, and this is something Mark actually added recently for me.

Data helps you make decisions faster.

If you can not as the CEO, have to have a conversation over something where you

could just run a test, your company is going to be able to move quicker,

because every decision won't have to come to you.

People on the team,

if they're clear on what that goal is, that north star metric is.

They can run the test, see if they moved the metric and

then only come to you when they know if they've moved the metric or not.

Decisions like that help you move faster.

Knowing what your most important priority is because you've looked at the data.

That helps you move faster.

Make sense?

Cool.

So now you want me to push as much of the time as possible so I'm pushing for

as much as time as possible.

So let's talk about tactics.

First 100 users.

Sam talked about this brilliantly.

If you actually listen I thought Tracy in the previous conversation

was just fantastic.

When she talked about getting 30 of her friends in the construction industry to

try Plangrid, and then asking them to pay and 29 of them kept paying.

Like ask your friends first for your first 100 customers.

Get people who will give you honest feedback, and

I love the point about make them pay, if it's a paid product.

Second, research and then reach out to who you'd like to have as your users.

Totally makes sense, right?

Like, you can do how Oscar was discovered by the Lob Team right.

Just email a bunch of people, don't get caught in a spam filter.

Don't just auto generate emails or you'll definitely get caught in a spam filter.

But send personal emails,

get introductions, see which friend you have in common.

I emphasize by the way, I have never start up, so I might be entirely the wrong

person to tell you about first 100 users, but Sam asked me to.

Third, social media and PR, PR is unscalable, but

it can give you amazing bumps.

Amazing bumps.

And in the early days I think PR is a good growth leader.

Later on, I think companies don't realize when PRs stop being a growth lever,

because they have these other metrics.

But you can get these great one-time bumps.

Just know you can't get them every single week, and sometimes they are a one off.

And finally, by ads.

I am not as against ads as Sam is, as a great method for growing users.

I think that buying ads can be absolutely fantastic.

I built my career buying ads, two of the most valuable internet companies,

Google and Facebook make all their revenue from selling ads like ads

are actually a good thing.

If you get the targeting right and we're going to talk about that in a second.

If you're having a growth team, if you're focused on growth tactics what people

talk about is growth hacking, though I hate the term If you pour fuel.

This is a picture from a guy called Nick on my team, Nick.

I did not tell him to hold his hand into the flames, which you'll see in a second.

If you pour lighter fluid onto dead coals with no spark, you get nothing.

And if there's one thing to take away from this lecture,

you need to have product market fit to drive growth.

You need retention to drive growth.

Otherwise, every growth tactic,

every acquisition tactic you could possibly run doesn't matter.

But if you have a spark there.

If you have a glowing ember you can pour fuel on the fire of your product and

that is what growth tactics does.

So I look at things based on channel.

Targeting creative and then conversion.

So you think about channel, I'm going to do some greatest hits here.

If ou look at SEO, you think, I have to think about keyword research.

Which keywords do I want to show up for on Google on the other search engines?

And make sure you have content targeted against those keywords.

You need to get links, links from the internet to your site.

If you're a Silicon Valley startup you technically get a lot of links off high

value sites so you usually don't have to worry about that.

But then you need to link inside your site.

Many people will tell you links don't matter anymore,

in my experience that's not true.

I don't know why it's not true, but it's not been true in my experience.

And then finally, there are a bunch of basics out there on SEO,

you can look them up.

There are a load of websites that talk about them.

STO book,

MOOs, there are a bunch of websites out there you can read about SCO tactics.

Four page search.

The biggest thing I see people do wrong is they dont think about incrementality.

They by page search and they don't necessarily say, well,

would I have got those customers anyway?

So look really hard.

If you're going to buy page search as a startup you should see a step change

in your line.

Your marketing team,

if you've reached the stage where you have a marketing guy should show you a line

that is flat and then they turn on page search and it goes up.

That's the same of any ads if you are a small startup.

If your a big company incrementality analysis there's this great Berkeley

professor Steve Tedelis whose done a deep study in it with Ebay that I love and

I think he's really smart.

Big companies, you have to be very, very smart about getting incrementally.

You're a small startup.

Show me a step change.

Do think about marginal not absolute ROI.

So people always look at I spent this much I got this much.

How valuable is it?

That's not what you want to look at.

You want to look at the last dollar you spent.

How much did you get for that?

Not overall does that make sense?

Not as many knots.

I'm going to blast through that.

Look at margin ROI, look at diminishing return curves and economics.

And then again, you need to do keyword research.

So email and SMS push notifications.

This is something every startup gets wrong.

We always optimize for ourselves.

Have you seen the Instagram thing on the internet where someone shows a phone of

an Instagram user with a million followers when they posted something and

they have push notifications on.

And the string just goes [SOUND] and

all the notifications are coming in at high speed?

That's what we're like for our own products.

For our products we are the power users and by nature

every start up I have gone and talked to has always optimized for themselves.

Has always optimized for the power user.

And so by optimizing for the power user, what you do?

Well, we shouldn't put down by default because it's going to completely expand my

whole screen.

Power users are smart.

If you give them the options, they know how to turn off notifications.

It's your marginal users you should care about with notifications.

That person for whom they received their first Like on Instagram or

the host that gets their first notification on Airbnb.

Or the seller that gets their first bid on Ebay or

the buyer who's just been out bid in their first item.

That's the person that needs to get the notification.

And needs to respond to it, and you need to optimize for in your head.

Marginal user is the number one thing in email, SMS,

push, notifications, and even offline mail which Rob was talking about and

is still very important for many companies.

People get wrong, they optimize for themselves.

Beyond that you need to make sure it gets delivered, opened, clicked on, etc.

There's so much more here and

if I push through all of it I'm not going to get to any questions.

Do you want me to get to questions?

>> This is pretty important.

How about this and no questions.

>> This and no questions, is everybody okay with that?

Everybody for next week.

>> So think about on site merchandising.

You own a lot of media.

There's this idea of earned, bought, and owned media.

Does that make sense?

So if you post on Facebook and

it gets loads of likes you've earned that media, right.

If you bought an ad on Facebook or google or somewhere, you've bought that media.

And then if they're on your own website, you own that media.

People underestimate the value of owned media.

If you have a lot of traffic on your site, ask them to invite their friends.

Ask them to complete that next action in your product.

Think about your on-site merchandising as an extension of your product, but

understand that on-site merchandising is a marketing problem.

So how can you trigger the right thing?

Someone has just bought an item for you, immediately like Amazon does.

Trigger for them a set of suggestions like why don't you buy this and this, so

the customers who bought this also bought this, works incredibly well.

What about creative?

Creative is really important.

It should be personalized, because they're on your site, you know their name.

Hey Alex, why don't you buy these items that other similar customers have bought.

Performs way better than buy these other items that similar customers have bought.

That's one simple personalization.

But instead of just saying this customer bought this item,

you should do a prediction.

This person, all the items they've ever bought let me predict for

them in a personalized way what item they should buy.

Look at Facebook and how often we say five of your friends have done x.

Social context, that's personalization.

Use people's own profile picture.

We have a great personalization example where when you're on a Facebook page,

we show you what your ad would look like if you bought it,

with a Buy This button underneath it.

It could not be more personalized.

On-site merchandising gives you so many opportunities to deeply,

deeply personalize the creative.

And finally, paid and organic social.

I do work for Facebook, we do make all of are money from ads.

And I personally think that I am not the best person to speak about buying ads on

Facebook, because I'm the guy who buys ads off Facebook to drive traffic to Facebook.

But what I'll say is if I left Facebook today, and I was a start-up

I would be crazy not to use our custom audiences and look-alike targeting.

So I highly, highly encourage you if you think about nothing else on Facebook ads

to look into those.

It's a great way for you to say these are my best customers.

Upload the list to Facebook.

How do I find more customers like those?

It works incredibly well for a lot of companies.

And the other thing across all social media is to a great extent you're

dealing with interesting squared.

Because the click-through rate on your thing, the like rate or

whatever on your thing will be related to how interesting it is, and

also the distribution, whether it's being shared via a messaging app, reshared on

Twitter, or on Facebook, will be driven by whether people found it interesting.

Make sense?

Cool.

Okay, targeting.

Behavioral is most important.

So what you did Is much more important than who you are.

There are 50 year old, 60 year old parents and

grandparents who don't need two friends to be active on Facebook.

And there are some who need 500.

And there are teenagers who need 500 friends and some who need two,

and you're much better to figure out like what is the behavior that tells you that.

Then say on a demographic basis, I'm going to do this versus this.

People who buying Nike's, Men's Nike trainers typically are men, but sometimes

people buy for their boyfriend, or their husband, or their dad or whatever.

And so if you just target women with women's trainers and

men with men's trainers you are making a mistake.

What you should do is look at what signals you to show the right ad or

the right content to the right person at the right time.

Targeting matters.

I've talked about this before.

Okay, creative.

Creative is important but

it's nowhere near as important as the above two unless you have a 10x creative.

If you produce the Dollar Shave Club video,

probably it's very very worth it but most people don't produce

a piece of creative that amazing that changes the world.

Typically I find targeting comes before creative in every single way, and

channel comes before that.

Does that make sense?

So creative is awesome and really important and can give you a big uplift,

but think about your channel first and your targeting second.

If you have creative though, you should put it in context,

you should personalize it, you give it a call to action.

On call to action, this is a great example.

Where on every ad back in the day on Facebook we had a little link at the top

that said advertise.

We translated it wrong by mistake in French.

We found this with the data in French, we translated to clear an ad.

Create an ad.

And we had significantly more,

actually 40% more acquisition from that channel that

was at the time our single most important channel for advertiser acquisition.

We took the French translation and

we reverse translated it and changed the whole world.

Instead of saying the kind of passive advertise to the active create an ad.

And we achieved a 40% bump in that most important channel for us for

advertiser acquisition.

That is the call to action, like what you say matters.

So when I say creative doesn't matter as much as channel, and

targeting, it still matters a lot.

And the words you use matter a lot.

This is the example I said about in context with intent.

If you are on your own page, this used to be ad we would show to you,

that say hey this is what your ad would look like elsewhere on the site.

And you can see that by hour this was the hour that we doubled

advertiser acquisition for Facebook with the weekly push.

If you want to see more about this, Brian Hale does a great talk about this at F8,

and it's online.

And then the last thing, yes.

The last thing is conversion.

Conversion is really really important, but

it's not important until you've thought about all the other things first.

So things to think about with conversion.

The landing page people land on will determine what your conversion

rate is going to be.

If you can, don't have one button on it that they then have to click

to go to the registration form.

Put the registration form in the landing page.

It's important to know what your most important metric is though.

Back at eBay, what we had was this original metric confirmed registered

users, and a new metric activated confirmed registered users.

And an activated confirmed registered user was someone who bought an item,

bid on an item, listed an item.

And a confirmed registered user was someone who registered and confirmed and

that was it.

To get confirmed registered users we wanted to pump all of our search traffic,

this was back in 2004 we were spending huge amounts on search.

We wanted to pump all of our search traffic at the registration page.

To get activated confirmed registered users we actually wanted to land it

on the search term that you were looking for.

So if you were looking for

a trampoline, show you the trampolines you could buy on eBay.

That change,

and we changed all of our advertising to change the landing page, dropped

confirmed registered users significantly that we were acquiring for eBay.

But it increased activated confirmed registered users.

It increased revenue and lifetime value per registered user.

And it increased total revenue per day from the people we were acquiring.

So although I think you can focus on the registration form really hard.

Any commas, sometimes it's really,

really important to land people on what they are searching for, so

that they will activate and they will buy on your service.

So it's not as simple as just put the registration form online.

But we did do that on Facebook.

So the final thing I like to say is Facebook moves fast.

And move fast is an absolute core value for us as a company.

Like core, core, core value and we still do bold things like taking messenger out

of Facebook, that was challenging at the time.

But I think now people are seeing how valuable a decision that was.

And that was a project that I was deeply involved with and ran.

We're willing to move fast, we're willing to be bold because fundamentally a good

plan violently executed on today is better than a perfect plan next week.

Data helps you move fast.

These growth tactics help you move fast.

Best of talks about making decisions with 70% of the data not 90% of the data.

And I deeply believe in that.

So move fast, focus on retention and

have one key metric that you care about for

your company's growth.

Thank you. >> [APPLAUSE]

>> Okay, apparently we have time for

one question.

I'm sorry.

Go for it.

>> So you mentioned that [INAUDIBLE].

How do you think that creating the.

>> So the question was for the magic moments, you can kind of tell how you can

figure out went they happened and that how you can create them.

I mean honestly if you look at LinkedIn,

Facebook, Twitter every social network in the known universe.

What we do is, we immediately ask you to import your contacts.

So once we know that friends is the thing that we need to do for

you, we focus on what is the way to remove friction from getting you friends.

If you look at the example I used with eBay, we hadn't really figured out that

activating you was important to generating the revenue long term.

So we shifted the landing page and the registration form to the search results.

So we got you to the item you wanted to buy as quickly as possible.

And that caused you to activate and get to that magic moment faster.

So those are the kind of things once you identify it

I actually think the tactics fall out very logically.

My boss Javi likes to say that common sense is the least common of all senses

and I agree with him.

But if you step back and just look okay, if I want more people to buy an item on

eBay should I land them on the registration page or

the search results page?

You'd probably land them on the search result page and

that would get them to that magic moment faster.

>> Thank you.

>> Thank you, okay.

>> [APPLAUSE]