Webinar: How to Be an AI Product Manager by Facebook AI Product Leader, Natalia Burina

Summary

本次演讲由Facebook的AI产品经理Natalia Barina主讲，她分享了AI产品管理的重要性和AI产品经理的角色。她强调AI在当今社会中无所不在，对决策产生深远影响。AI产品经理需要理解业务问题，制定战略和路线图，并确保AI产品的公平性、隐私性、透明度和可靠性。她还讨论了AI产品管理与传统软件产品管理的不同之处，包括处理AI的不确定性和建立实验文化。最后，她给出了成功作为AI产品经理的三个建议：讲好故事、制定清晰的计划和快速上手新工作。

Takeaways

• 🤖 AI产品管理的重要性日益凸显，因为AI正在以前所未有的规模做出关键决策。
• 🚀 AI产品是自动系统，通过从数据中学习来做出面向用户决策，机器学习使计算机系统通过现有数据训练获得学习能力。
• 🛠️ AI产品经理的角色是为公司最重要的商业问题构想和实现有影响力的AI解决方案。
• 🎯 识别AI可以解决的正确问题和优先级是AI产品经理的关键任务。
• 🔍 AI产品经理的类型包括专注于产品的、平台的、研究的以及专注于构建负责任AI的产品经理。
• 🌟 AI软件开发与传统软件开发最大的不同在于其概率性，机器学习将工程从确定性过程转变为概率性过程。
• 📈 AI产品经理需要掌握的实用技能包括使用创新工具、理解AI产品管道、以及关注业务目标和技术潜力。
• 🔧 构建负责任的AI意味着确保AI的公平性、隐私保护、可解释性、可追责性和可靠性。
• 📊 成功作为AI产品经理需要深入理解问题、内部化业务目标、对齐和跟踪正确的指标，并培养实验文化。
• 🚦 面对AI技术的强大力量，产品经理必须意识到伴随的责任，并始终以不伤害用户和客户为出发点。
• 🌐 作为AI产品经理，应与所有利益相关者合作，特别是领导层，以设计和对齐适当的指标。

Q & A

AI产品经理的主要职责是什么？

AI产品经理的主要职责是设想和实现对公司有重大影响的AI解决方案，解决最重要的商业问题。他们需要首先理解AI能解决的最重要的商业问题，其次识别和优先处理这些问题，并制定愿景、策略和路线图。最后，他们需要确保这些解决方案得以实现。

AI产品与传统软件产品在产品管理上有何不同？

AI产品的开发是概率性的，与传统的确定性过程不同。机器学习将工程从硬编码的算法和规则转变为基于大量输入输出对的统计学习过程。这意味着AI产品经理需要适应这种不确定性，并理解模型预测可能会有一定的错误率。

AI产品经理需要具备哪些技能？

AI产品经理需要具备包括使用设计思维工具（如UX模型、线框图、用户调查等）在内的多种技能。他们需要理解AI产品管道，包括数据、模型、服务基础设施和实验等环节。此外，他们还需要理解机器学习可以解决的问题类型，如排名、推荐、分类、回归、聚类和异常检测等。

如何确保AI产品的公平性和无偏见？

为确保AI产品的公平性和无偏见，产品经理需要设计和优化AI以避免内置的偏见。这意味着要确保AI的决策基于伦理和包容性原则，同时要意识到数据可能编码了人类和社会的偏见，并采取措施来减轻这些偏见的影响。

AI产品经理如何处理AI产品的隐私和安全性问题？

AI产品经理需要在构建AI时保护用户隐私，并确保系统能够抵御攻击。他们需要考虑AI可能被欺骗的方式，并采取措施来增强系统的鲁棒性，以防止通过操纵输入数据来破坏算法。

AI产品经理如何确保AI产品的透明度和可解释性？

AI产品经理应确保AI的决策过程是透明的，并且能够解释AI为何做出特定的决策。这对于高风险的使用案例尤其重要，因为它们可能会直接影响人们的生活。透明度可以通过提供决策背后的逻辑和数据支持来实现。

AI产品经理如何确保AI产品的问责性？

AI产品经理需要建立检查和平衡机制，进行验证和合规性审查，确保有人类监督。这意味着AI决策过程不仅要自动化，还要能够在必要时进行人工审查和干预。

AI产品经理如何确保AI产品的可靠性和持续高性能？

AI产品经理需要持续监控AI系统的性能，并及时检测和解决数据问题，以防止模型性能衰退。这包括对系统进行持续的优化和调整，确保AI产品能够持续提供高质量的服务。

AI产品经理如何衡量AI产品成功与否？

AI产品经理需要与所有利益相关者，特别是领导层，合作设计和对齐适当的度量标准。AI应该支持业务度量标准，并展示其对业务的影响。成功的衡量可能包括早期的胜利、业务指标的提升以及与业务目标的一致性。

AI产品经理如何培养实验文化？

AI产品经理应该帮助组织建立成功使用AI所需的文化。这包括鼓励智能风险的承担、从失败中学习并不断迭代。他们应该从简单开始，逐步获得早期胜利，并慢慢扩展AI产品的范围。

AI产品经理在开始新工作时应该采取哪些策略？

AI产品经理在开始新工作时可以采用安德鲁·博斯沃斯的冷启动算法。这包括安排与关键人物的30分钟会议，介绍自己，提出必要的问题，记录详细的笔记，并询问下一步应该与谁交谈。这种方法有助于快速了解新环境并建立联系。

Outlines

00:00

🤖 人工智能产品经理的角色与挑战

本段落介绍了Natalia Barina作为Facebook的AI产品负责人的背景，并探讨了人工智能在当今社会中的重要性和影响。AI产品是自动系统，通过学习数据来做出面向用户决策的机器学习技术，使得计算机系统能够通过在现有数据上的训练来学习，并基于之前未见的数据做出预测或其他结果。Natalia强调了AI产品经理的职责，包括理解AI可以解决的重要商业问题、制定愿景和战略路线图，并确保AI产品的责任性、公平性、隐私性、可解释性和健壮性。

05:01

📊 机器学习与AI产品开发的实践技能

Natalia讨论了AI产品经理在实践中需要的技能，包括确定期望结果、交付方式和产品使用方式。她强调了在AI产品开发过程中理解数据、模型、服务基础设施和实验的重要性。Natalia还提到了AI产品生命周期，包括数据和训练、设计和开发、验证和测试、部署批准以及监控和优化。此外，她还提到了AI产品管理与传统软件产品管理的不同之处，特别是在机器学习的开发过程中的不确定性和概率性。

10:02

🛡️ 构建负责任的AI产品

在这一段中，Natalia强调了构建负责任的AI产品的重要性，包括确保AI的公平性、隐私保护、透明度、可解释性和健壮性。她提到了AI可能带来的潜在问题，如偏见、数据泄露和对抗性攻击，并强调了AI产品管理者需要理解这些风险并采取措施来最小化它们。她还提到了AI产品管理的一些关键领域，包括确保AI不带有偏见、保护用户隐私、提高AI的透明度和可解释性、确保AI的可追责性以及建立可靠的AI系统。

15:03

🚀 成功的AI产品经理的策略和建议

Natalia分享了成为成功的AI产品经理的策略和建议。她强调了识别正确问题和解决方案的重要性，以及深入理解AI可以解决的问题。她还提到了内部化商业目标、理解技术的能力和局限性、以及理解潜在的危害。Natalia提出了与所有利益相关者合作以设计和对齐适当的指标的重要性，并强调了建立实验文化的重要性。她还分享了三个产品管理的技巧：讲述好故事、制定书面计划以及快速启动新工作的策略。最后，Natalia以感谢听众的参与结束了她的演讲。

Keywords

💡人工智能产品经理

人工智能产品经理（AI Product Manager）是指负责规划和管理人工智能产品的专业人员。他们需要理解并解决重要的商业问题，通过AI技术提供解决方案。在视频中，演讲者Natalia Barina就是一名AI产品经理，她分享了自己在Facebook支持一个约50人团队的经验。

💡机器学习

机器学习（Machine Learning）是人工智能的一个分支，它使计算机系统能够通过学习已有数据来提高性能。在视频中，演讲者提到机器学习让工程师从确定性过程转变为概率性过程，通过训练数据对模型进行训练，使系统能够对未见过的数据做出预测。

💡深度学习

深度学习（Deep Learning）是机器学习中的一种算法，它通过模拟人脑神经网络结构来学习数据的表示和特征。在视频中，演讲者提到深度学习被用于推断驾驶员的疲劳或分心程度。

💡AI产品生命周期

AI产品生命周期是指从AI产品概念提出到最终部署和优化的整个过程。这个过程包括数据收集、模型训练、设计开发、验证测试、部署批准以及监控优化等阶段。

💡公平性与偏见

公平性与偏见（Fairness and Bias）是指在AI产品开发过程中，确保AI系统不会因数据中的偏见而导致不公正的结果。AI产品经理需要识别并减少偏见，以确保AI系统的决策是公平和无歧视的。

💡隐私保护

隐私保护（Privacy Preservation）是指在开发和使用AI产品时，保护用户个人信息不被泄露或滥用的措施。AI产品经理需要确保AI系统在收集和处理数据时，遵守隐私法规并采取安全措施。

💡可解释性

可解释性（Explainability）是指AI系统能够向用户清晰解释其决策过程的能力。这对于高风险的应用场景尤为重要，因为用户需要理解AI做出特定决策的原因。

💡责任归属

责任归属（Accountability）是指AI产品在决策过程中应有明确的负责方，确保有检查和平衡机制，以及人类监督。这涉及到AI产品在出现问题时，能够追溯并采取相应措施。

💡可靠性

可靠性（Reliability）是指AI产品在长期运行中保持高性能和稳定性的能力。AI产品经理需要确保模型不会随着时间的推移而性能下降，并且能够及时发现并解决数据问题。

💡实验文化

实验文化（Experimental Culture）是指鼓励尝试、容忍失败并从中学习的组织环境。AI产品经理需要帮助构建这种文化，因为它与传统软件开发不同，AI奖励那些愿意承担智能风险的个人和公司。

💡业务指标

业务指标（Business Metrics）是指用于衡量公司业务绩效的量化指标。AI产品经理需要确保AI项目支持业务指标，并展示其对业务的影响。正确选择和跟踪业务指标对于AI产品成功至关重要。

Highlights

AI is making consequential decisions at an unprecedented scale, becoming ubiquitous in various sectors.

AI products are automated systems that learn from data to make user-facing decisions.

Machine learning enables computer systems to learn from existing data and make predictions based on new data.

Examples of AI in everyday life include spam filtering, Google searches, and self-driving cars.

AI product managers (AIPMs) will be essential as AI becomes more prevalent in all technologies.

The role of an AI product manager is to envision and realize impactful AI solutions for business problems.

AI product managers come in different types, including product-specific, platform, research, and responsible AI focus.

AI software development is probabilistic, unlike traditional deterministic processes.

AI PMs must understand the AI product pipeline, which includes data, model, serving infrastructure, and continuous experimentation.

AI can cause unintended consequences if not carefully designed and optimized.

Responsible AI development involves ensuring fairness, privacy, robustness, explainability, and accountability.

AI product managers should focus on identifying the right problems and solutions, aligning with business objectives, and understanding potential harms.

AI should support business metrics and show its impact on the business.

Fostering an experimental culture is crucial for AI product managers to succeed.

AI projects can be difficult, and it's important to start simple, achieve early wins, and grow gradually.

Telling a good story, having a written plan, and using the cold start algorithm are tips for AI product managers.

Transcripts

good afternoon everyone

um

my name is natalia barina and i'm an ai

product lead at facebook

where i support a team of about 50

engineers pms and designers

i'm excited to bring you the event today

so let's talk about how to be an ai

product manager

why are we here today well ai is making

consequential decisions at an

unprecedented scale

in the last decade ai has become

ubiquitous

driving increasingly complex and

consequential decisions like credit

approvals college admissions courtroom

bail decisions etc

ai products are automated systems that

learn from data to make user facing

decisions

machine learning gives computer systems

the ability to learn by being trained on

existing data

after training the system can make

predictions or deliver other results

based on data it hasn't seen before

some examples of ai today that we don't

think about or we don't notice but spam

filtering in your email box

google searches

self-driving cars of course this one is

visible if you see them

driving around if you happen to be in an

area where they have them

um there's also systems that use deep

learning to infer how distracted or

tired drivers are

there's so many more

one of my favorites is the estimation of

house prices

because all technologies are touched by

ai soon every pm will be an aipm

this is really impossible to escape we

will all need to understand ai because

it's becoming so omnipresent

so let's talk about

what is the role of an ai product

manager

what is an ai product manager and what

do we do

well

an ai product manager

works to envision and realize impactful

ai solutions for companies most

important business problems

um

and the way to do this is to first and

foremost understand what are the most

important business problems that ai can

solve ai is not always appropriate in

all situations

secondly it's to identify

and

what are prioritize the right set of

problems and develop a vision strategy

and roadmap so this is really getting

into

what it is that you need to do

and then third is all product managers

our job is to make it happen

now there are different types of ai

product managers

um there are pr ai product managers who

work specifically on

products and need to have

um

a very

very strong understanding of what the

product does in order to use ai to

supercharge that specific product

um there are ai product managers who

work on a platform

so they work on developer tools and

infrastructure in order to build ai

there are ai product managers who work

on research so this is working with ai

researchers to bring research

breakthroughs to production

typically these are

product managers

in bigger companies where there's a

whole department that just focuses on

research

and then finally

there are uh product managers who focus

on building ai responsibly although

building ai responsibly has to be

everyone's responsibility but building

ari responsibly what does that mean

ensuring that ai is fair

private

robust explainable and then there's

accountability um

to people who use the ai

how is

product management

different

uh in the ai than for other um

for regular software

well

let's talk about this this is a really

interesting one

um

ai software developments

is

probabilistic

so this is the biggest difference

between traditional ai traditional and

ai tech products

machine learning shifts engineering from

a deterministic process to a

probabilistic ones

this means that instead of writing code

with hard-coded

hard-coded algorithms and rules they

always behave in a predictable manner

ml engineers collect a large number of

examples of input and output pairs and

use them as training data for for their

models

with machine learning

we often get a system that is

statistically more accurate than simpler

techniques but with a trade-off that

some small percentage of model

predictions must always be incorrect

sometimes in ways that are hard to

understand

[Music]

this is why it's it's sometimes hard for

for people to make that shift it's a

very different way of building products

let's talk about practical skills for ai

product managers

what is it that you need to uh

what kind of skills do you need to build

in order to become an ai product manager

well

as with um all all uh product managers a

pm must determine what is the desired

outcome how that outcome will be

delivered and how the product will be

used

before

starting the process of building which

every time you start it's a process

that's expensive

and it's an investment

so at the very beginning in the ideation

phase ai product managers should be able

to use the same rapid innovation tools

that

design experts use including ux mockups

wireframes user surveys

at this stage it's really critical to

frame the problem or the opportunity

that the product addresses

there's

sort of different

categories of problems that ml can um

can solve in their ranking

recommendation

classification regression clustering

anomaly detection etc

um it is imperative

that every ai pm should understand the

ai product pipeline and what does this

mean well you need

to build an ai product you need data you

need a model you need serving

infrastructure

and you have to experiment until ai

works and gives you the outcomes that

they're satisfactory

in reality there are many candidate

models

that are created during the development

process but basically the ai life cycle

is

this diagram i have here

data and training

design and development validation and

testing is important

approval to deploy and then finally you

have to have monitoring an optimization

to ensure that even when it's built and

it's out there in the wild things change

and so it's important to monitor the

system for quality

continuously

with great power comes great

responsibility so

ai supercharges technology products and

it's extremely powerful new technology

and here i'm borrowing from from

spiderman though this actually goes way

back to antiquity

um

we as ai product managers have to

understand that ai is so powerful

that we have to really think about the

responsibility that comes with

developing such products

in medicine there's the hippocratic

oaths stating first do no harm

likewise we as ai practitioners should

as a starting point not harm our users

and customers what do i mean by that

well

unless carefully designed and optimized

ai can cause unintended consequences

ai gets better

with more data but data encodes human

and societal biases

some examples are facial recognition

that cannot recognize darker skinned

faces

ai that inadvertently recommends high

paying jobs to men and low paying jobs

to women

etc there there are so so many ways that

ai can go wrong

let's get a little bit more specific

about areas for responsible ai

development and what you should think

about when you're building ai

oops

um so

first and foremost

by the way these are not in any

particular order so i shouldn't say

first and foremost but ai

you have to make sure that ai is fair

and it does not have bias so that it's

built in a way that's ethical and

inclusive

secondly it's important to preserve

people's privacy when you're building ai

and to secure secure against attacks um

there's some really interesting ways

that ai can be tricked

uh by in image recognition for example

where you just do a little bit of

manipulation

um

of an image and it completely can break

the algorithm so it's very important to

think about the adversarial aspects um

when building ai

ai should be transparent

so it should

we should be able to understand the why

behind the ai and why it makes certain

decisions

this is especially important when they

are

high stakes use cases that impact

people's lives

um

[Music]

excuse me all right

um ai must be accountable so there must

be checks and balances validation and

compliance

and be able to to uh really be

have human oversight and then finally ai

should be built in a way that is

reliable so you have uh continuous high

performance that the models don't decay

you detect data issues this is where

monitoring um in our pipeline up there

is really important

so how do you succeed as an aipm um

what are how do you um

how do you make this work it's hard

um

the most important thing is to be able

to identify the right problems and the

right solutions for them and the way to

do that is to deeply understand what are

the problems

um that ai can can solve

for for people who use it

secondly you have to internalize

business objectives um so

you have to understand what is

what is really the point of your

business and how you can support the

business with ai remember you have to

supercharge the business with

ai you have to understand what is the

technology what it can and can't do ai

is not suitable in all cases it is not a

silver bullet

and then as an ai product manager you

have to understand the potential harms

of this powerful technology

you have to align and track

align on and track the right metrics um

actually coming up with the right

metrics is not trivial it's oftentimes

difficult for a business to define and

agree on metrics

ai should support business metrics and

should show how

it makes an impact to the business

of course the worst case scenario is

that there aren't any business metrics

or that

sometimes what you see is there are

business metrics and then there are ai

metrics and they are in no way connected

and it's very difficult

uh

to to

to see

how they work with each other so

all ai should ladder up to the greater

uh business goals

and to get there you have to work with

all stakeholders but especially

leadership

to design and align on appropriate

metrics before building your ai products

without clarity and metrics it's

impossible to be successful and

how do you do this we could we could

have a whole session on this it is not

straightforward and it is highly

dependent on what kind of ai you're

building and what kind of a product you

are supporting

um

as an aipm it's really important to

foster an experimental culture so

part of an ai product manager's job is

helping the organization build the

culture it needs to succeed with ai

because it's so different from

traditional software development where

the risks are more or less well-known

and predictable

ai really rewards people and companies

that are willing to take intelligent

risks

um

and and for this you have to be willing

to fail everything will not work

but it is important to document the

lessons and to keep iterating

machine learning gives companies real

competitive advantages in prediction

planning sales in almost every aspect of

this business

however even simple machine learning

projects can be difficult and building

ai is much harder than most people

realize

um

there's there's some data to show from

from venture b uh venturebeat claims

that 87 of machine learning products

never make it into production

and harvard business review says the

first wave of corporate ai is bound to

fail so

machine learning

is not a silver bullet

the ability to make decisions based on

data is

a prerequisite for an experimental

culture

and measurement obsessed companies every

part of their product experience is

quantified and adjusted to optimize user

experience

um so you have to start simple get early

wins

wow i am getting

apology about that

um

start simple get early wins and slowly

grow your ai speed of products

as a conclusion i want to leave you with

my three favorite pm

pro tips

and they are telling a good story is

everything

having and telling a good story is the

most important thing you can do as a

leader

and this is

this is important for every pm

uh but it's especially important for

aipms because we have to make

people understand what it is that we do

oftentimes it's very technical it's hard

to explain

um

second

one of the things that i have been

implementing for some time in my career

is each half i have a written plan for

beginning

at each

beginning of each half and i have a

written plan for how next six months

will go

i share this with leadership and

colleagues and it really helps keep me

focused i never take meetings that don't

align with with my goals and

it's a it's a good way to

to really stay

balanced and make sure you're doing the

right things and then finally if you're

starting a new job one of the things i'd

recommend is

andrew bosworth's cold start algorithm

and this is a way that will help you

ramp up faster than you ever have before

and it's basically saying um if you

start a new job schedule

30 minute meetings with key people and

each 30-minute meeting should be

you should introduce yourself ask

everything you need to know

[Music]

um take copious notes and then ask who

else you should talk to it's pretty

simple but it's extremely effective

with that i thank you so much for your

time today i enjoyed giving this

presentation and sharing my experience

in ai product management

uh should you like to please reach out

to me on twitter this is my twitter

handle

thank you so much

[Music]

you