Principles For Human-Centered AI | Michael I Jordan (UC Berkeley)

Summary

The speaker advocates for a market-driven approach to AI, emphasizing decision-making and economic principles over human imitation. They argue that intelligent systems should create value by connecting producers and consumers, like Amazon and Uber, rather than just mimicking human intelligence. The talk highlights the potential of machine learning in creating robust, enjoyable human experiences by focusing on market mechanisms and data-driven decision-making.

Takeaways

• 🚀 The future of business models should focus on becoming more like Netflix, Uber, or Amazon by leveraging intelligent systems and market mechanisms.
• 🤔 The term 'intelligent systems' is often used without a clear definition, prompting the need to look at Earth from a Martian perspective to identify what is truly intelligent.
• 🧠 Traditional AI has focused on mimicking human intelligence, but this may not be the most effective approach for creating robust and beneficial systems.
• 🏙️ Cities and markets demonstrate a form of intelligence through their ability to coordinate complex systems for the distribution of goods and services.
• 🛠️ The new frontier in computer science is not just about what happens inside the computer, but also how computers interact with the world through market mechanisms.
• 🚗 The concept of 'autonomous' vehicles is flawed; instead, vehicles should interact with a larger system, similar to air traffic control, for better coordination.
• 🔢 Machine learning has evolved from backend data analysis to customer-facing recommendation systems, and now to pattern recognition and decision-making.
• 💡 The speaker advocates for a shift in AI research from human imitation to creating systems that make better decisions in the context of markets and scarcity.
• 💼 Businesspeople should consider machine learning as having two parts: pattern recognition and decision-making, with the latter being crucial for creating value.
• 🎵 The music industry is an example where a market exists but is not fully monetized, suggesting opportunities for new business models that connect producers and consumers.
• 🌐 The potential for AI is vast, extending beyond traditional areas like music to art, cooking, and personal services, where connecting providers with consumers can create jobs and wealth.

Q & A

What is the main focus of the talk given in the transcript?

The talk focuses on the business model of the future, discussing how to become a company like Netflix, Uber, or Amazon, and the importance of technical fields driving business strategic level thinking.

Why does the speaker suggest considering oneself as a Martian computer scientist?

The speaker suggests this perspective to encourage out-of-the-box thinking and to look for inspiration for intelligent systems on Earth that could be mimicked or improved upon in computer algorithms.

What is the speaker's view on the current approach to artificial intelligence and its focus on imitating human intelligence?

The speaker believes that the focus on imitating human intelligence is misguided and that a more promising approach involves creating market mechanisms and systems that interact with the world in a more integrated way.

What does the speaker consider as an example of intelligence on Earth that could inspire computer algorithms?

The speaker considers the market systems, like the coordination of ingredients and supplies in a city's restaurants, as an example of intelligence that operates efficiently and could inspire computer algorithms.

Why does the speaker argue that the term 'autonomy' in the context of self-driving cars is incorrect?

The speaker argues that 'autonomy' is incorrect because self-driving cars should not operate in isolation like humans but should interact with all other cars in a more integrated system, similar to air traffic control.

What is the speaker's opinion on the current state of machine learning and AI in terms of economic impact?

The speaker believes that while machine learning and AI have had significant impacts, especially in backend operations and recommendation systems, the next generation of AI should focus more on decision-making and market mechanisms for greater economic value.

What is the concept of 'two-way markets' as mentioned by the speaker?

The concept of 'two-way markets' refers to a system where both sides of a transaction can see and bid on each other, creating a more efficient and dynamic market that reduces noise and increases the likelihood of successful transactions.

How does the speaker suggest using recommendation systems in a new way to create value?

The speaker suggests using recommendation systems not just to suggest items but to facilitate real-time bidding and transactions, such as recommending a customer to a restaurant and allowing the restaurant to offer a discount in return.

What is the speaker's view on the potential of market-based AI and its impact on jobs and the economy?

The speaker believes that market-based AI has the potential to create jobs and generate wealth by connecting producers and consumers directly, rather than relying on advertising or other indirect monetization methods.

What are some of the technical problems the speaker is working on to advance market-based AI?

The speaker is working on problems such as multiple decisions, markets under uncertainty, cloud-edge interactions, and provenance, which are essential for powering the next generation of market-based AI.

Can you provide an example of how the speaker suggests applying market-based AI to the music industry?

The speaker suggests creating a platform that not only streams music but also provides data to musicians about their popularity in different cities, enabling them to negotiate performances and create a sustainable income, thus fixing the broken music market.

Outlines

00:00

🚀 The Future Business Model and AI's Role

The speaker addresses CEOs, discussing the future of business models, inspired by companies like Netflix, Uber, and Amazon. They question the common understanding of 'intelligence' in AI, urging the audience to consider alternative forms of intelligence observed in systems like city markets, which are robust, reliable, and adaptable. The speaker suggests that the real innovation lies in market mechanisms and the interaction of computers with the world, rather than just imitating human intelligence. The talk emphasizes the importance of strategic thinking at the intersection of technology and business.

05:01

🤖 Rethinking AI: Beyond Human Imitation

The speaker critiques the current focus on imitating human intelligence in AI, arguing for a broader perspective that includes decision-making and market dynamics. They discuss the evolution of machine learning from backend processes to customer-facing recommendation systems, which have significantly impacted industries. The speaker advocates for considering machine learning as having two parts: pattern recognition and decision-making, with the latter being crucial for real-world applications and economic value generation.

10:02

🌐 Creating Two-Way Markets with AI

The speaker introduces the concept of using AI to create two-way markets, exemplified by a recommendation system for restaurant seats. They propose a system where customers' preferences are broadcasted to nearby restaurants, allowing them to bid for the customer's business, thus creating a dynamic market that reduces noise and increases efficiency. The speaker suggests that this approach can be applied to various economic goods and services, creating new business opportunities and revenue streams through transaction fees.

15:02

💡 Empowering People with Market Mechanisms

The speaker argues for a shift from trying to understand and predict human desires through data to empowering people through market mechanisms. They propose a model where data flows lead to economic value and transactions, creating happier customers and a more honest way of making a profit through transaction fees. The speaker also touches on the social benefits of creating jobs by connecting people and providing services, suggesting that this approach is the next big opportunity for businesses.

20:03

🎵 The Music Industry as a Market Model

Using the music industry as an example, the speaker illustrates how a market-based approach can revitalize an industry where traditional models have failed. They propose a system where musicians can see data on their popularity in different cities and use this information to negotiate performances and other opportunities, thus creating a sustainable income. The speaker suggests that this model can be applied to various domains, emphasizing the potential for wealth creation and job generation.

🔬 Research Directions for Market-Based AI

The speaker concludes with a discussion on research directions for developing market-based AI systems. They highlight the importance of addressing multiple decisions, markets, uncertainty, and the interaction between cloud and edge computing. The speaker also mentions the need for algorithms that can handle competition, such as bandit algorithms in the presence of multiple decision-makers, and the importance of controlling false discovery rates in large-scale A/B testing.

Keywords

💡Intelligent systems

Intelligent systems refer to the concept of creating machines or computer systems that can perform tasks that would normally require human intelligence, such as understanding language or making decisions. In the video, the speaker challenges the audience to consider what 'intelligent' means in this context, suggesting that mimicking human intelligence may not be the ultimate goal for future business models.

💡Markets

Markets, as discussed in the video, are systems where buyers and sellers interact to exchange goods and services. The speaker highlights the intelligence inherent in markets, noting their robustness and reliability, and suggests that understanding and mimicking market dynamics could be a more fruitful approach to creating intelligent systems than simply trying to replicate human intelligence.

💡Autonomy

Autonomy, in the context of the video, refers to the idea of self-governance or independence, particularly in the development of self-driving cars. The speaker argues that the concept of autonomy is misguided when applied to cars, suggesting instead that they should be part of an integrated system that interacts with other vehicles, similar to air traffic control.

💡Machine learning

Machine learning is a subset of artificial intelligence that involves the use of algorithms to parse data, learn from it, and make informed decisions. The speaker mentions that machine learning has been crucial to the success of companies like Amazon and Netflix, but also emphasizes the need to move beyond pattern recognition to decision-making in the context of markets.

💡Recommendation systems

Recommendation systems are algorithms that suggest items or content to users based on their preferences or behaviors. The speaker critiques the conventional use of recommendation systems for movies or books and proposes a more dynamic, market-based approach where recommendations can lead to transactions and create economic value.

💡Economic perspective

The economic perspective mentioned in the video involves considering the principles of microeconomics, such as supply and demand, when designing intelligent systems. The speaker argues for the importance of this perspective in creating systems that can make decisions in a market environment, taking into account scarcity and competition.

💡Scarcity

Scarcity refers to the basic economic concept where the demand for goods and services exceeds the available supply. In the video, the speaker uses the concept of scarcity to argue for the importance of decision-making systems that can allocate resources effectively in situations where not everyone can be accommodated.

💡False discovery rate

False discovery rate is a statistical measure used to evaluate the expected proportion of false positives among the results from a hypothesis test. The speaker discusses the importance of controlling the false discovery rate in large-scale decision-making processes, such as online advertising, to ensure the reliability of the system's recommendations.

💡Two-way market

A two-way market, as described in the video, is a system where both buyers and sellers can interact and influence each other's decisions. The speaker gives the example of a recommendation system for restaurant seats, where customers can signal their preferences and restaurants can bid to attract those customers, creating a dynamic and efficient market.

💡Gig economy

The gig economy refers to a labor market characterized by the prevalence of short-term contracts or freelance work as opposed to permanent jobs. The speaker suggests that the next wave of the gig economy could involve connecting producers and consumers directly through intelligent systems, creating new opportunities for employment and wealth generation.

Highlights

The talk focuses on the business model of the future, aiming to inspire CEOs to think like Netflix, Uber, or Amazon.

Intelligent systems are often misunderstood; the speaker suggests viewing them from a 'Martian' perspective for fresh insights.

The speaker argues that human brains, while intelligent, may not be the best model for Martian-inspired computer architecture due to their Earth-specific adaptations.

Cities and their complex systems, like San Francisco's restaurant supply chains, are presented as examples of distributed intelligence and robustness.

Markets are identified as inherently intelligent systems that have been driving human happiness and viability for thousands of years.

The importance of microeconomics and game theory in understanding market intelligence is emphasized.

The speaker criticizes the current focus on imitating human intelligence in AI, suggesting a shift towards creating systems that interact with the world.

The concept of 'autonomy' in self-driving cars is challenged, proposing a more integrated system similar to air traffic control.

The evolution of machine learning from backend operations to customer-facing recommendation systems is outlined.

The potential of machine learning to create new billion-dollar industries through pattern recognition is discussed, but with skepticism on its economic value.

The speaker introduces the idea of machine learning having two parts: pattern recognition and decision making, with a focus on the latter.

The importance of considering the consequences of decisions made by machine learning systems is highlighted.

Recommendation systems are reimagined not just as tools for suggesting products, but as mechanisms for creating two-way markets.

The potential of using recommendation systems to create dynamic markets in various industries, such as music, art, and cooking, is explored.

The speaker proposes a new formula for AI involving data, algorithms, and markets, moving away from human imitation.

The social and economic benefits of creating jobs and wealth through market-based AI are discussed.

Examples of companies like Amazon that understand the importance of connecting producers and consumers are given.

The speaker shares his research interests in areas such as multiple decisions, markets, uncertainty, and cloud-edge interactions.

Innovative projects in bandit algorithms and asynchronous online false discovery rate control are mentioned, showing the future of market-based AI.

Transcripts

all right I'm pleased to be here so this

talk is a little different than the one

I usually give this is a talk not aimed

at the mathematicians in the audience so

much as the CEOs so what I'm going to be

trying to convey is what I consider the

business model of the future how do you

become a Netflix or an uber or an Amazon

what's missing out there so how is it as

our technical field drive business

strategic level thinking so we all claim

that we're working on intelligent

systems it would be great if computers

were more intelligent whatever that

means but that's kind of the issue what

does that mean we just use that word as

if we all know what we're talking about

so extract yourself from the earth for a

minute go up to Mars and imagine you're

a Martian computer scientist and you

have a very primitive computer

architecture and you're looking down at

earth to try to get inspiration for how

to make it better all right so you look

down and say what's intelligent on the

planet earth that I could try to mimic

and put inside my algorithms and my

computers so the first thing you might

notice is these things brains and minds

and you might agree that there are some

how intelligent whatever it means on the

other hand you might note that they're

pretty well adapted to earth and maybe

the intelligence that are exhibiting is

pretty adapted to earth and not so

adapted to Mars or to other situations

moreover understanding brains and nert

webs of neurons and how that leads to

thought and behavior seems really

challenging and in fact I argue that

even on planet Earth where we get this

look at and probe at a quick pretty

closely we're very very far away from

understanding in fact we don't have

really a glimmering yet of how thought

arises from webs of neurons all right so

as the martian computer science just

kind of lost what else down on earth is

intelligent that you could algorithm a

size and all right well it's not that

hard if you just think out of the box a

little bit and I don't think enough

people are thinking out of the box so

think about a city like San Francisco

and think about every restaurant

and think about every dish that the

restaurants creating it has ingredients

and those ingredients have got to arrive

at the restaurant every day and it

happens there's a web of decisions being

executed distributed slightly

coordinated worth individuals deciding

to bring you know their goods from one

side of the city the next and so on and

it works all those restaurants get what

they need and every home gets what it

needs more or less and it doesn't work

perfectly that's always interesting but

it works really well works at all scales

small big scale City it works for 3,000

years

okay and it's probably done more to make

human happiness and life viable than

anything else in the world so it's

markets markets are intelligent by any

definition they're robust they're

reliable they work in all conditions

rain or shine day in and day out and we

know some of the principles it's called

microeconomics okay it's game theory

it's trade its markets and so on we

don't know all the principles but that's

what it's interesting about it

what new principles are needed if we

bring that into the world of the

computer I know what does it mean to

bring in the world of computer well if

it was just inside the computer that's

maybe not so interesting that's like

resource allocation or whatever fine but

we're no longer just inside the computer

that's what's new about computer science

is that it's out in the world and in the

computer in the bridge is the hard part

and if you just still think about the

old computer science mentality and build

a service in the computer and then make

it so good that people are attracted to

it like you know moths to a flame and

then understand everything about the

people just by being inside the computer

that's kind of the world we're in and

that's kind of not working very well

you've got to have market mechanisms

where the computers interacting with the

world in the sense of an actual market

alright there's another way to say this

people have been too focused on this

notion of imitating human intelligence

that's what AI referred to in the 50s

and now that we brought that terminology

back to describe machine learning people

think that's what we're still working on

is imitating humans so that we can take

out a driver in a car and put in a

computer just like the human or a

dialogue person we put in a human a

computer right no that's not the really

this kind of intelligence we can it she

in our lifetime that will make human

life more robust more fun more pleasant

and so on

think about autonomous driving cars that

word autonomy is just dead wrong

you don't want self-driving cars to be

like a human

that's autonomous and you put it in the

car in the car is suddenly autonomous

right no if that car should interact

with all the other cars around it in a

much deeper way than we currently do

right if one car knows that boys just

ran out into the street every other car

around that should know immediately okay

it should be more like the air traffic

control system it should be an

integrated system okay autonomy is just

the wrong way to think about the whole

thing all right and even intelligence is

not embedded in the car itself and

intelligence is better in the entire

system right I've given this slide

before I think in fact it's SPARC last

year so I won't spend too much time on

it but just look at a little bit of the

history of this field that I called

machine learning for many years that

people are now calling AI you know it's

not new already in the 90s the backend

was extremely important a fraud

detection search supply chain management

so a company like Amazon would have

never become Amazon without doing

machine learning in the backend at very

big scale with big data and lots of

computers and lots of Engineers they did

that and they became Amazon I think the

next generation was noticing that all

that data analysis could be turned not

just at the back end but turned towards

the customer and recommendation systems

emerge so you saw a great talk about

that just now that was already the

second generation and then the Netflix

and Amazon kind of led that those two

generations led to billion-dollar

industries

okay no one talks about it certainly the

journalists don't talk about it but that

had a huge impact on our world this

third generation which I think it was

just really pattern recognition finding

patterns and data you know the success

stories are things like speech

recognition of vision oh yeah I don't

see the economic value yet

okay now many of you're working on that

it'll be great

but I don't see the billion dollar

industry yet okay because you're kind of

just taking out a human and putting in a

computer and it's not so third of me

you're gonna you know make the world a

better place or you're gonna make a lot

of money doing that

okay not so clear and in fact I think

what's really emerging is this web of

decisions these markets that a computer

with data analysis can create if you

think in that way and that's what I want

you to think about as a businessperson

all right so instead of thinking about

pattern recognition is the big

achievement and the big technology

tensorflow if you will think about

machine learnings having two parts to it

pattern recognition and decision making

all right now decision making what is

that well isn't that just taking the

output of tensorflow and threshold in it

okay well no all right so think about

things that have consequences so I go

into the doctor's office and you measure

a lot of things about me my DNA

my blood pressure and all that you know

hundreds of thousands of variables you

put that into a big neural net and it

says you have liver disease you need to

have your liver transplant it tomorrow

urgently what does it mean it says that

well there's some number that's over

some threshold point seven and it's 0.71

all right is that a decision okay well

no I'm not gonna just stop there I'm

gonna say wait a minute what data are

you using to make that prediction is

that data recent is it about people like

me what if I were to exercise more or

what if I did this or what I did you

know this thing about me or about my

past or whatever etcetera so I want a

whole dialogue and that's just for one

decision okay but one that has

consequences real-life decisions have

consequences we want to talk about

Providence where the data came from

counterfactuals and have a whole

dialogue all right but decisions are

never in isolation we're always making

decisions in the context of other

people's decisions other agents there's

going to be scarcity we also have to

worry about false discovery rate if I'm

gonna have a system make a hundred

thousand decisions today like uber who's

allocating cars to places in the city

right I want most of that Baguio

decisions to be good ones if I think

about them independently that's no good

I want the overall set of them to be

good okay moreover I really want to

think about sets of decisions by sets of

agents over time and I want to see as

time goes on your air rate is not kind

of wildly fluctuating it's it's

controlled in some nice way and I want

asynchronous AV testing is kind of a

I'm aware we use data to make good

decisions over time by the large set

collections of agents who are loosely

coupled that's the real-world problem to

me it's not just taking an image of a

computer you know on a computer or

taking and saying there's a kangaroo in

the image decision okay all right and

even taking it further when we have

competition and scarcity we really need

to have more than just making decisions

with thresholds and reinforcement

learning and all that we need an

economic perspective okay okay so let's

look at recommendation systems which you

heard about in the previous talk from an

expert you all know what they are you

keep a record of customers purchases if

customers were similar you recommend

items from one customer to another

customer and very important you know

let's call it a billion-dollar industry

most companies are doing this it's

become a commodity you can download

software to do it and so a lot of people

look at that and say great I can build a

company because this off the platform

exists I can get the computers I can get

the image I can download I can run a

recommendation system for something else

than movies all right and build a

company all right if you do that you're

gonna run into all kinds of trouble

because you're not thinking out of the

box you're thinking about the old model

recommending movies or books or whatever

right what if I recommend the same movie

to everybody well everybody's happy how

will you stream the bits to everybody

else care city same thing with books

nowadays you can print it on demand the

Amazon can get it to you within three

days even if they recommend the same

book to everybody right but now if you

build a system that recommends other

kind of things that are economic goods

like restaurant seats I come out on out

of the you know a taxi in Shanghai at

the end of a datum they have meetings I

don't know Shanghai very well I don't

speak Mandarin I'm by myself I want to

have a recommendation for a place to go

eat okay but not just a recommendation I

just a list or a set of advertisements

or Yelp pages or something like that I

want to have a button where I push it

and I say geo locate me now broadcast to

all the restaurants around me that I'm

here

make it a recommendation system meaning

that you see something about me that I

have a certain price point

I liked Szechuan cuisine etc etc let the

restaurants see that and let them bid on

me that restaurant says we like you

the next 10 minutes will give you a 10%

discount if you push the button we just

created a two-way market and both sides

are seeing the other through a

recommendation system all right so

that's gonna cut down all the noise all

right I'm not gonna see 500,000

restaurants with Yelp reviews and some

on the other side they're gonna see

customers that are viable and as soon as

we make a transaction we fill a seat in

the restaurant you're not going to

recommend that same seat to a hundred

thousand people okay simple but very

important now the company that does this

has just built a two-way market how do

they make their money well it's way

easier than build an advertising Empire

rather you take 5% of your cut of the

transactions all right and you can make

big money that way if this is really

rolled out throughout Shanghai you're

gonna make tons of money and then

worldwide you know great what if I were

to recommend streets to drivers

everybody needs to get to the airport if

I recommend the same Street to everybody

it's no longer a good Street now this is

obvious right but people don't tend to

think this way they roll out their

service and then maybe a thousand people

are using it and everybody's happy man

the whole city starts to use it and

everybody's not happy anymore oh oh you

should have thought of that right but

people are not doing it now why are they

not doing it well because they have this

AI mentality all right let's just take

the kind of Facebook example they think

that I'm gonna figure out what you want

by looking at your browsing history so

if I'm going to the airport today

all right there's several options maybe

I'm really in a rush and I really want

to go on the fastest street and I'll pay

a little bit more for that all right

whereas you're going to the airport and

you think about it for me and say hey

I'm not in such a rush I'll go on a

slightly slower Street and I'll save

some money for some pre next day that's

a little market kind of mechanism right

you could build something like that and

people could have a little interface

where they could start to start to say

and everybody be happier the whole thing

would flow faster right Silicon Valley

instead is thinking in the following way

we're gonna figure out so we're gonna do

a big dynamic program and figure out

where each person wants to go and then

we're gonna load balance and decide who

goes down what Street how are we gonna

figure that out well we know your

browsing history what you know

that's that's insane but that's where

we're at people think that you're gonna

see everything about us and then offer

us the best things that we really really

want even don't even know it all right

that mentality is coming from too much

of a wedding of advertising and company

all right

instead we want to empower people to be

connected to each other producers and

consumers uber does this so it's not a

mystery you can do this writers and

drivers does uber have to advertise to

make money you know no they can take a

transaction fee all right so diners are

on one side of the market restaurants

are on the other you don't have to do

any advertising at all you just connect

them and then you take a cut drivers are

on one side of the Market Street

segments are on the other the street

segment bids for me to go across the

street all right so it's not just

classical microeconomics that I'm

arguing for here that is part of it but

it's also at scale and it's a

recommendation system mentality that we

have a lot of data that doesn't tell us

what people want directly all right but

it gives us some honing down so they can

have a straight ability simple market in

which they participate in and one that's

adaptive okay so if we're gonna have a

formula let's call AI data plus

algorithms plus markets critically and

let's leave behind the human imitative

side of this it's fine if you figure out

how the brain works great and build a

company on that I just not gonna I'm not

gonna invest but if you think you've got

it fine

but I think if you build market

mechanisms I'm ready to invest because

there's so much sitting there on that

that's not being monetized in any way so

wherever you have data flows instead of

thinking of that as just how do I get

the data to flow faster and how do i

monetize that with advert or with

something or advertising think about if

data is flowing probably some economic

value is flowing on the on top of that

and if I really connect up on both sides

and make transactions appear on top of

that probably everyone's going to be

happier really like my service not be so

annoyed about privacy issues because

they're getting value out of it and

probably I can take a transaction fee

which is a perfectly honest way to make

a living okay so for AI research this

autonomy and human imitative side I

think is his is needs to be diminished

we need to think more about federated

agents that interact with each other in

the world of scarcity IT business

instead of building a service that tries

to figure out everything about humans

and give them what you think they want

and make money from an artificial market

between you and advertisers directly

connect people to each other all right

so the social consequences of this

you're gonna be able to create jobs when

you connect people to each other you've

essentially created a job the provider

for provides something to somebody and

they pay that's the job all right

and now we can what kind of domains can

we do this in and this is where I'm

getting closer to the business model all

right so I'm gonna give an example which

I like to give it's music we all think

that music is like hot you know it's

everywhere you know it's being streamed

and everything where they hear music

well know anybody who's a musician I'll

tell you they cannot make money being a

musician Beyonce can but that's um

that's not a market that's a monopoly

all right all right there are tons of

people and probably many of you do it on

the weekend you go home and you make

music on your laptop and you uploaded to

SoundCloud a lot of people drive taxis

in the week and they do that on the

weekend and the music's really good if

you go to SoundCloud it's so good that

companies like Spotify and all streaming

it to people and tons of people listen

to it in fact most music me listen to in

this very moment is music done in the

last four years and it's done by people

who just put it up on soundcloud it's

not Beyonce at all so somehow there's a

wonderful market city there but it's not

a market no one's making money off of it

now we don't have a job doing that all

right well how can you fix this instead

of just creating a streaming platform

for music and then monetize it by

advertising well it's not that hard it's

really easy be a data scientist so

anybody who's putting music on

soundcloud it may be over some threshold

of popularity at the end of the week

gets to see a dashboard alright and that

dashboard has that on every city in

whatever country you're in and you see

how many people listen to you in each

city alright so if my musics popular

maybe and you know Cleveland Ohio 10,000

people listen to me last week you know I

didn't know that now that I know that

I'm gonna go to the venue owners and

Cleveland show them that data and then

they're gonna say wow you're popular

here can you give a show here I'll say

yeah I fly out there I give a show I'm a

twenty thousand dollars if I do that

three times during the year I have a

salary I could quit my taxi job which is

about ready to disappear

anyway and I could be a musician and

moreover now that I know who listens to

me and comes to my show that's easy too

you know I just have a little QR code on

the screen while giving the show I can

make them other offers I'll say I'll

come plate your daughter's wedding for

you know $20,000 $10,000 we'll have a

little negotiation right that's a market

that's gonna create so much wealth and

then the company providing all this

right this is not just a little

peer-to-peer thing the company has to

provide this and do it well takes a cut

of all that okay rich you can make me a

lot of money doing this all right that's

just for music think about art thinking

about cooking thinking about other

personal services of all kinds so this

is kind of the share of the gig economy

if you will all right but I don't think

people are getting how big this really

is this is the next trillion-dollar

thing to do this I think that Amazon

gets this and I think that a lot of what

you see all the innovation from Amazon

right now

okay it's not just because they're so

damn smart it's because they see that

they're trying to provide services to

people and connect producers and

consumers they get that all right I

don't think Facebook gets it they think

they're supposed to just connect people

and make a community of whatever that

means

okay

so I hope that inspires some of you to

think about market metals in your own

business now I'm a researcher so well

first of all I do want to mention

there's a company already doing this

they beat you to it it's called United

masters go online so there's some famous

musicians behind this who themselves

make tons of money but they don't like

the system and they think it's broken

and they want to fix it so they're

paired with computer scientists to build

something called United masters but

there's still funds tons of opportunity

that net1 discreetly the company that

survives okay now I'm a researcher I

work on technical problems in this area

this is a list that I've been using for

the last few years kind of summarizing

some of them you know I work on multiple

decisions I work on markets uncertainty

cloud edge interactions abstractions

provenance and all this

it sounds like boring back-end stuff if

you will or computer science II kind of

stuff there's no sexy AI language on

there but to me these are the problems

to solve and this is good to power the

next generation of market-based AI and

so I have slides which I'm going to kind

of skip a little bit but I just want to

tell you I 30 seconds left here here's

some projects if you'll go to my website

you'll see some of the emerging work on

this topic we've been working on banded

algorithms bandits are a really great

thing to think about your machine

learning person it's not reinforcement

learning it's not supervised learning is

kind of in-between it's that you have

several options and no one's telling you

which is the best option so you have to

try them out a little bit so a be tests

have this flavor so we're doing that

when there's competition what if I have

multiple people doing a bandit algorithm

and there's competition only one of them

can have the arm that they if they all

select the same arm only one of them

gets the arm okay brand new problem

obviously big implications for real

world problems finding Nash equally with

Grady based algorithms and high

dimensional accent spaces this blends

machine learning style thinking with

microeconomics thinking and then

asynchronous online false discovery rate

control if you're doing in large-scale

ad testing in any company you want to

have false discovery rate control you

want it to be asynchronous and you want

to be online and those words were not

thought to be possible to achieve in the

past they are possible I'm going to just

mention

two of my colleagues Tiana zernich and I

dindt around us we have a paper that's

now in the archive where you will see an

algorithm that does this it's a pretty

simple little economic based algorithm

where the decision makers are all given

a little bit of wealth and as they make

decisions they lose some wealth but when

they make certain kind of decisions they

gain back some wealth and it means you

can make decisions in a whole company

over a lifetime and I could stop you at

any moment and say how many errors have

you made up until now and that rate will

always be under control so this is a

brand new word it's part of me the

broader problem is she learning when

you're doing decision making ok that's

how it slides on that should I'm going

to skip because I've reached the end of

my talk that may move all the way to the

end in just a couple of slides here okay

if you don't control false discovery

rate you will make bad errors and this

kind of gives you an example of making

bad errors where you have type 1 type 2

error control really really good but

your number of false discoveries is

really really terrible ok so this is a

topic I hope everyone has learned about

where we'll learn about so here we go

party comments all right so let me just

pop back up to the right top this is the

error where everyone's focused on

pattern recognition it's being a

commodity you know tensor flow pipe

torques or whatever you will that's all

that's what they do it's pattern

recognition it's not decision making and

I hope you've seen a little perception

here of how important decision making is

for research going forward for this to

really be a field and for people to have

business plans which are not so quite

quite so broken thank you very much

[Applause]