How Statistics Supports our Intuition | Ostap Okhrin | TEDxDresden

Summary

The speaker explores the omnipresence of statistics in daily life, from avoiding tigers at the zoo to planning vacations and playing board games with kids. They delve into weather forecasting, analyze movie ratings on IMDB to understand gender preferences, and even apply statistical analysis to J.R.R. Tolkien's works. The talk illustrates how everyone has an 'internal statistician', making decisions based on probabilities and data, and emphasizes the pervasive role of statistics in our lives.

Takeaways

• 📊 Statistics is a branch of mathematics and probability theory, linking complex mathematical formulas with the real world full of data and observations.
• 🧠 Everyone has an 'internal statistician' that influences daily decisions, such as avoiding crowded places on Saturdays or choosing the best time to refuel a car.
• 💏 Statistical tendencies are observed in human behavior, like the common direction of head turns during kissing, and who initiates kisses in heterosexual couples.
• 🌡 Weather forecasting is an example of statistics in action, using various models to predict temperatures and probabilities of different weather conditions.
• 📈 The speaker compared different weather forecasting websites, noting how they use statistical models to provide short-term and long-term predictions, which can vary due to different underlying models.
• 🎲 Board games can demonstrate the role of statistics in determining the probability of winning and losing, and how the distribution of outcomes can be symmetric or skewed.
• 👫 The importance of considering the sample size and demographic when evaluating ratings, such as movie ratings on IMDB, to understand potential biases or dependencies.
• 🎬 The use of statistical analysis to determine if a movie is 'male' or 'female' based on the correlation between gender and ratings, and the importance of a balanced rating distribution.
• 📚 The application of statistical techniques like cluster analysis to determine the similarity between books, as demonstrated with works by J.R.R. Tolkien and his son Christopher Tolkien.
• 📘 The difference in perception and content between books written by J.R.R. Tolkien and those edited by Christopher Tolkien, highlighting the role of posthumous editing in shaping literary works.
• 🌳 The idea that children can 'feel' statistics through games, indicating that an understanding of statistics is innate and can be developed through everyday experiences.

Q & A

What is the role of different disciplines in decision-making during a day?

Different disciplines like biology, physics, and chemistry support us in making decisions by providing knowledge about the natural world and its phenomena. For instance, biology might warn us about the danger of approaching a tiger, while physics helps us understand forces needed to cross a river.

How is statistics related to mathematics and probability theory?

Statistics is a branch of mathematics and is deeply connected to probability theory. It links complex mathematical formulas with real-world data and observations, helping us make sense of patterns and trends in the data.

Why do people tend to avoid shopping on Saturdays according to the speaker's experience?

People tend to avoid shopping on Saturdays because it is commonly known to be crowded, suggesting that there is an internal statistician in everyone that guides decisions based on observed patterns and experiences.

What does the speaker imply about the statistical tendency in car fuel prices?

The speaker implies that car fuel prices are higher during early mornings and evenings, and lower during the day, suggesting a statistical understanding of pricing patterns that influences when people choose to refuel.

What is the significance of the direction people lean while kissing according to the speaker?

The speaker mentions a statistical tendency for people to lean their heads to the right while kissing in most cases, and that in heterosexual couples, males often initiate kisses, indicating how statistics can reveal subtle patterns in human behavior.

How does the speaker describe the process of checking weather forecasts for vacation planning?

The speaker describes checking weather forecasts by visiting different websites like 'avatar' and 'vet online', which use various statistical models to predict temperatures, helping in making informed vacation plans.

What is the game 'Asteroids Garden' and how does it relate to the concept of probability?

'Asteroids Garden' is a game involving rolling dice to collect fruits or advance a crow in the garden. It is a game of pure chance with an 80% winning probability against the crow, illustrating the concept of probability in a fun context.

How does the game 'Lifter des Plaines' differ from 'Asteroids Garden' in terms of decision-making?

'Lifter des Plaines' also involves rolling dice but differs in that it is a competitive game where players make individual decisions based on the dice roll, contrasting with the team-based, decision-less gameplay of 'Asteroids Garden'.

What is the significance of the correlation coefficient in the context of movie ratings and gender?

The correlation coefficient is used to measure the dependency between movie ratings and gender. A higher coefficient indicates a stronger relationship, suggesting that certain movies may be preferred by specific genders based on the ratings.

How did the speaker use statistical analysis to compare books by J.R.R. Tolkien and his son Christopher Tolkien?

The speaker used cluster analysis to compare the most frequently used words in the books by J.R.R. Tolkien and Christopher Tolkien, resulting in a dendrogram that grouped the books based on their linguistic similarity, revealing differences in their writing styles.

What conclusion does the speaker draw about the presence of a 'statistician' within us?

The speaker concludes that everyone has an internal statistician that helps us make decisions based on patterns and probabilities, whether in everyday activities like shopping or in more complex scenarios like analyzing literary works.

Outlines

00:00

📊 The Ubiquity of Statistics in Daily Life

This paragraph discusses how various disciplines, especially statistics, influence our daily decisions. It uses examples such as avoiding close encounters with tigers due to biological instincts, understanding physical forces when crossing a river, and the role of chemistry in modern medicine. Statistics, a branch of mathematics and probability theory, connects complex mathematical formulas with real-world data and observations. The speaker illustrates this with everyday examples, such as choosing shopping days to avoid crowds and understanding fuel price fluctuations. Furthermore, the speaker embarks on a 'statistical journey' through a day, starting with checking weather forecasts for vacation planning, comparing different models and their probabilities, and ending with family activities influenced by the day's weather.

05:00

🎲 The Role of Statistics in Board Games and Movie Selection

The speaker explores the application of statistics in board games and decision-making processes, such as choosing which movie to watch. They describe playing 'Asteroids Garden' and 'Lifter des Plaines' with their children, analyzing the probability of winning and the games' average completion time. The games are categorized as 'pure gambling' with no strategic decisions involved. The speaker also discusses using IMDB ratings to decide on a movie, considering the distribution of ratings and the correlation between gender and movie preferences. They use statistical analysis to determine if a movie is gender-dependent in its appeal, ultimately choosing 'It's a Wonderful Life' as a film suitable for all, based on its even rating distribution and lack of gender bias.

10:04

📚 Statistics in Literature: Tolkien's Works Analyzed

In this paragraph, the speaker reflects on the presence of statistics in literature, specifically in the works of J.R.R. Tolkien. They mention reading not only the well-known 'The Hobbit' and 'The Lord of the Rings' but also lesser-known works such as 'The Silmarillion', 'The Children of Húrin', and 'Unfinished Tales of Middle-earth'. The speaker notes that the latter three were edited by Tolkien's son, Christopher, after his death. Using statistical analysis, they compare the most frequently used words across all twelve books, employing cluster analysis to create a dendrogram that visually groups the books by their lexical similarity. This analysis reveals that the books edited by Christopher Tolkien are distinct from those written by J.R.R. Tolkien, highlighting the subtle differences in style and content.

Keywords

💡Statistics

Statistics is a branch of mathematics that deals with the collection, analysis, interpretation, presentation, and organization of data. In the video, it is portrayed as a tool that helps in making decisions by linking complex mathematical formulas with real-world data and observations. The script uses examples such as predicting weather and analyzing movie ratings to illustrate how statistics can guide everyday decisions.

💡Probability Theory

Probability theory is a mathematical framework that deals with the analysis of random phenomena. It is foundational to statistics and is used to make predictions and inferences about data. The script mentions that statistics is a part of probability theory, indicating its role in understanding the likelihood of various outcomes, such as weather forecasts or the success of a board game.

💡Data

Data refers to pieces of information, often numerical, collected through observation and used for analysis. The video script emphasizes the importance of data in the application of statistics, as it is the raw material that statistical methods process to extract meaningful insights, such as predicting weather patterns or understanding public opinion on movies.

💡Forecasting

Forecasting is the process of making predictions about the future based on past and present data. In the script, weather forecasting is a central theme, demonstrating how statistical models are used to predict future conditions. The speaker compares different forecasts to show how statistical analysis can lead to varying predictions.

💡Decision Making

Decision making is the cognitive process of selecting a course of action from multiple alternatives. The video script discusses how statistics supports decision making by providing insights based on data analysis. Examples include choosing the best day for shopping to avoid crowds or selecting a movie to watch based on ratings and reviews.

💡Board Games

Board games are a type of game that involves counters or pieces moved or placed on a pre-marked surface according to the rules. In the script, board games serve as an example of how statistical concepts like probability and winning chances can be understood and experienced in a fun, tangible way with family.

💡IMDb

IMDb, or the Internet Movie Database, is an online database of information related to films, television programs, and video games. The script uses IMDb as an example of how statistical analysis can be applied to understand movie ratings and public opinion, highlighting the importance of considering demographic factors like gender in the analysis.

💡Contingency Table

A contingency table is a type of table in a matrix layout that displays the frequency distribution of variables. In the video, the speaker constructs a contingency table to analyze the relationship between gender and movie ratings, demonstrating a practical application of statistics in understanding dependencies between variables.

💡Correlation Coefficient

The correlation coefficient is a measure that expresses the extent to which two variables are linearly related. In the script, the speaker calculates the correlation coefficient to quantify the relationship between gender and movie ratings, providing a numerical measure of the strength and direction of the association.

💡Dendrogram

A dendrogram is a tree-like diagram often used to illustrate the arrangement of the results of hierarchical clustering. The script mentions the use of a dendrogram in a cluster analysis of books by J.R.R. Tolkien, showing how statistical techniques can reveal relationships and groupings based on the frequency of word usage.

💡Internal Statisticians

The concept of 'internal statisticians' refers to the idea that everyone has an innate ability to make judgments and decisions based on informal statistical reasoning. The video script uses this term to emphasize that statistical thinking is a natural part of human cognition, as seen in everyday activities like choosing the best time to shop or play games.

Highlights

Different disciplines support decision-making in various scenarios, such as biology advising against approaching a tiger due to its predatory nature.

Physics helps in understanding forces needed to cross a river, illustrating the application of scientific principles in everyday life.

Modern medicine relies on chemistry, emphasizing the interconnectedness of scientific fields.

Statistics, a branch of mathematics and probability theory, links complex mathematical formulas with real-world data and observations.

Everyone has an 'internal statistician' that influences daily decisions, such as choosing shopping days to avoid crowds.

Statistical knowledge can inform decisions like the best time to refuel a car based on price fluctuations.

Statistical tendencies are observed in social behaviors, such as the common direction of head turns during kissing.

A journey through statistics in a day, starting with planning vacations and checking weather forecasts, demonstrates the practical use of statistical models.

Weather forecasting websites use statistical models to predict temperatures, with variances highlighting the nature of probabilistic predictions.

Playing board games with children can reveal insights into winning probabilities and the role of chance versus strategy.

The game 'Asteroids Garden' is analyzed for its winning probability, showing a high chance of success against the game's challenge.

In contrast, 'Lifter des Plaines' has a lower winning probability and is more dependent on random outcomes, affecting game length.

IMDb ratings and their distribution can indicate the appeal of a movie, with statistical analysis revealing demographic preferences.

A contingency table and correlation coefficient can be used to determine the dependency between gender and movie ratings.

The speaker identifies 'It's a Wonderful Life' as a movie with no gender dependency in ratings, suggesting universal appeal.

Statistical analysis of word frequency in Tolkien's works reveals differences between books edited by J.R.R. Tolkien and his son Christopher.

Hierarchical cluster analysis organizes Tolkien's books based on relatedness, showing a distinction between original and edited works.

The speaker concludes that statistics are omnipresent and can be felt even by children through games, emphasizing the innate understanding of statistical concepts.

Transcripts

it's a well-known fact different

disciplines are supporting us in making

different decisions during our date

right you won't come very close to the

tiger in the zoo because the biology

says it's a predate or it might bite you

you also somehow feel the forces which

you need to show this tone in order to

reach the other bank of the river from

the physics right the modern medicine

would be partly possible without

chemistry and what about statistics how

does the statistics supporting

orientation formerly statistics is a

part of mathematics right and if you

look more deeper it's a part of the

probability theory it's a science which

is linking beautiful and the very

complex mathematical formulas with the

beautiful and the very complex real

world full of data and observations and

actually all of us have some internal

statisticians inside right you if you

have a chance you would barely go

shopping on Saturday because you know it

you'll be crowded there anyhow so you

better use Thursday or Tuesday also you

won't take tank up your car early

morning or in the evening because the

prices are higher you better use the

time during the day if you have a chance

and even statistically while kissing we

are bending over head to the right in

most cases and in the heterosexual

couples males are in more cases

initiating kisses right let me bring you

to the small journey with the statistics

over one day ok so imagine it's early

morning Saturday I just had my morning

coffee kids still sleeping wife is

sleeping so we are planning our

vacations

what is the most important in vacations

to weather right so what you hear you

first open the website you check out the

weather for the coming one or two weeks

that actually what I did two weeks ago

on the 16th of August right I opened

avatar come they are actually using a

variety of different underlying models

their statistical models machine

learning models metallurgical models

then they are very nicely brought up

together and delivered us as the

forecast with this nice picture right we

have a band laurel bond of this band is

telling us the forecasted temperature

over the night for the coming two weeks

and the upper bond is telling us their

forecasted temperature during the day

right it's obviously star decreasing

because autumn is coming winter is

coming but it's not the only website you

check out right there is a lot of them

and I did the same so I checked the vet

online right we see that actually short

term forecast there pretty precise they

are dancing both together but the father

goes into the future they're going apart

it doesn't mean that they are wrong

using different models but all the

predictions in the statistics they're

all working with the probabilities right

nowadays we see that actually the

Vatican forecast was better at that time

when I was making this forecast when I

was taking this data it looks like well

that accom thinks that during a day it

will be called cooler than Vetter online

sinks will be during the night okay I

spent too much time planning forecast

for the next two weeks so let's plan

today's already pretty late kids will

wake up and I should perform something

with them so let's check the weather for

today Oh doesn't look that nice in the

morning so during it day is also not

that fascinating and even in the evening

it doesn't look so nice although this

picture is pretty cool right we see we

have all the reservoirs in one picture

we have Sun if you have rain you have

cloud and you have a light pink over

there right it's a perfect prediction

for us but it we can look out of the

window and yeah it's raining there so we

should stay at home and what we can do

with the kids we play some table games

right you love playing table games we

love playing board games and they have a

lot of them and why are you playing with

the kids

you realize that some of them they are

very interesting it's love playing them

more and more and more and more and some

they are not that fascinating and this

is the question they raised to myself

and I was playing several games with my

kids so first game we took is the

asteroids garden the first garden

and see someone you know this game so

the rules are very very simple you have

a dice you have four trees with

different colors you're just throwing

your dice and if you have the color of

some fruit - just picking up this fruit

and if you have the crow then the crow

is making step forward into the garden

right pure gambling the kids making no

decision in this case they're throwing a

dice and making those steps and second

saying is it just a team game I cannot

win against my son he cannot win against

me we as a team winning against the CRO

or vice versa and we are both happy when

we ran and we are both sad when we don't

such a thing is that actually the

winning probability against the crew is

almost 80% it means in most cases we are

winning against the CRO rather than

losing an another stop the game is

pretty short we're doing just 22 steps

on average to finish the game well

imagine you're just 2 seconds for a

throw it within a minute you are done

and this average makes sense because the

distribution is very nicely symmetric we

have this pick in the middle they have

maximally for these tabs minimally 5

steps then we took another game at the

lifter des plaines - as a game in which

you are somehow generating random

numbers between 1 and 6 by throwing a

dice we've got this fancy machine

standing over there and your feature is

making step forward or jumping over some

fields are jumping over some fields

backwards right again pure gambling no

decision they just throw a dice and you

check out on which field you should land

and it's not a team game right because

I'm playing against my son and he or he

is playing against me so the winning

probability in this case is 1 over the

number of players

it means if you're playing two of us the

winning credibility is 50% if the whole

family is playing then else in this case

is 25% which is much much slower if it

take the case of the just ops garden

right and the game is a little bit

longer you should take 40 steps on

average to finish it right it's thing is

throwing throwing the dice even more

over this average is meaningless in this

case because the distribution is highly

skewed it means you can throw up to 200

times in order to finish the game right

10-15 minute setting is throwing your

dice okay it's time for a midday sleep

I'm bringing my kids to bad

and then we should perform something

with my wife let's check heart out the

window what's the weather outside it's

raining right so the resin forecast at

this point was pretty okay so we decided

to watch some movies there a lot of them

right different genres different types

different lanes and we wanted to wish

something again my wife told me a

friends of mine told me that the very

nice movie we should watch should be the

Twilight the the Breaking Dawn Part 1 I

said well but none of my friends and

nothing about this film then we decided

okay let's check out on IMDB what's the

rating there it's a beautiful website on

which here check the information about

all the films about the director about

the crew etc so it's 4.9 it's not that

high but it's okay you can watch this

but we a statistician both of us let's

check out what's the population of

waters telling about this film right and

we see it's have two picks at one at a

10 we're telling us it's 15 percent

telling the movies not that good 15

percent telling the movie is gorgeous

and we have some bump in the middle but

it's not what's interesting me meeting

them what's their boys are thinking

about the film in my age category right

so that's what I did here right so we

have a pick around 25% of boys thinking

that movies not really worth watching

and then I start declining but if you

check out for the female and that age

generation of my wife it looks slightly

their friend right so it makes me I mean

it gives us already some suspicion

thinking that maybe there is some

dependency between the gender and the

rating right so what we do is split up

the ratings into two groups bad rating

from one to five good from six to ten

then you construct the so called

contingency table and which allows us to

compute the correlation coefficient if

it's closer to one exist the perfect

dependence is closer to zero there is a

in dependency between two variables in

this case rating and the gender if you

keep it it for this case you obtain 0.47

which now tell us that there is a

moderate dependency between the gender

and the rating and we can conclude that

this is rather the latest movie so

what's a matter not so boys after

football told me that the pretty cool

movie to watch would be the middle man's

from 2009 I think my wife immediately

thought okay I made the same the same

investigation as you did

so immediately went to the IMDB the

rating is a little bit higher it's six

point six we immediately afterwards to

look what the problem would the other

people's telling about this film it's

more symmetric there is a bump in the

middle we also have it point to some

higher ratings and she immediately said

I would like to check out what the boys

of your a generation thinking about it

film looks like this it's not gorgeous

it's not wonderful movie but it's okay

right it's rating seven so it's a good

film worse watching although there is my

slight bump on the rating of two but

then she decided let's check out what

girl thinking about it still and it

looks like this right so we have a high

pick at the rating of two we then made

also the idea that actually there should

be some dependency between the rating

and the gender so we constructed again

the contingency table and the

correlation coefficient was 0.52

with indicating guys that it's actually

is a dependency between the gender and

the rating and we can conclude there is

a rather man movie and actually strongly

believe that those two movies are the

most female movie in the IMDB database

and the most man movie IMDB database so

then we should come somehow to the

consensus right so we decided to watch

it's a wonderful life by Frank Capra

and you know it has the high ranking for

8.6 it belongs to the IMDB top 250 and

if you look at the at the voting's you

see they ever very close to each other

for man as vandals from women and if the

movie is good there shouldn't be any

dependency for gender and if you will do

the same study as we did before you have

realizes there is no dependency between

the ratings and the gender so we can

call it movie for all so kids slept well

check out what the weather outside oh

it's sunny right again the weather

forecast was a kid first he said it

should be rainy then we have sunny day

so then we played outside you know Elba

play different games and then we need to

read some books right we also have a

variety of them you can read you can

choose from and my famous writer is the

genius of John Ronald Reuel Tolkien

right who created for us the world of

middle-earth wrote for us The Hobbit

the Lord of the Rings you definitely

know the writer you definitely saw the

movies I like I read all of them I

enjoyed reading them and them recently I

found three more books also written is

by John Ronald Reuel Tolkien

Silmarillion the children of whoring and

unfinished tales of middle-earth and

actually the children of húrin have been

published in 2007 right John Tolkien

died in 1973 and and children of húrin

published 2007 then little written there

it has been edited by Christopher

Tolkien I read the books they're very

nice but they are different right you

feel that it they are not like the books

by John Tolkien they're not worse

they're not better they just somehow

different then I have realised that

actually John Tolkien left a lot of

notes afterwards after his death what

Christopher Tolkien brought them

together fill all the gaps made this

story very smooth but what statistics

add about it why did it I pick up all

the books nine books by Tolkien three

books by the Christopher Tolkien and the

compute computer at the table twelve

books twelve columns and I've get out

250 most frequently used words in all

twelve books okay very simple words I

removed all own words like Dwarfs Hobbit

Gandalf Aragorn just simple words like

said will be take two etc etc simple

words and then plug it into the class to

analysis simple techniques that you

learn with our master students so class

analysis is producing us the tree which

we call the dendrogram the claw there

are books together in this tree the more

related they are so at first step you

realize that all the

Lord of the Rings coming together then

we have the Hobbit over there then the

rest of the books by the John Ronald

Reuel Tolkien and in the end three books

edited by Christopher Tolkien coming

into play so what I wanted to say with

this speech is actually the kids can

feel statistics right with the games

they know what's good was bad we have a

feeling of the statistics there is a

statistician living inside of all of us

and actually statistics is everywhere

thanks a lot

you