Optical illusions show how we see | Beau Lotto

Summary

This talk explores how our brains perceive the world, focusing on color and context. Through visual illusions and interactive experiments, it shows how perception is shaped not by direct sensory input but by how the brain interprets patterns based on past experiences. The speaker demonstrates that our perception of reality is subjective, influenced by context, and constantly redefined. The presentation highlights the adaptability of the brain and the importance of uncertainty in understanding how we interact with the world around us.

Takeaways

• 👁️ The game demonstrates that context is everything in perception, even when viewing identical objects under different conditions.
• 🎨 Color perception is a key example of how context shapes our reality, as seen in the example of the jungle scene where color reveals the hidden predator.
• 🧠 The brain processes sensory information by learning patterns and associations, not by interpreting sensory input in an absolute way.
• 🔄 Sensory information is inherently ambiguous and can have multiple meanings; the brain interprets this information based on past experience.
• 📊 Our perception is shaped by statistics from past experiences, which is why we fill in missing information in familiar patterns, such as reading incomplete text.
• 🔍 Even simple brain functions, like seeing color, can quickly adapt and redefine what we perceive as normal based on new information.
• 🎭 Perception is not just about the stimuli themselves but also about the meaning we associate with those stimuli, influenced by past experiences.
• 🐝 Even small creatures like bumblebees can perceive illusions and solve complex puzzles, showing that perception mechanisms are widespread in nature.
• 🎵 Visual information can be translated into sound, enabling people to navigate the world through hearing, and even compose music from visual stimuli.
• 🤔 The talk challenges the idea of certainty in perception, encouraging the audience to embrace uncertainty as a path to deeper understanding.

Q & A

What is the purpose of the game presented at the beginning of the script?

The game is designed to illustrate how context affects our perception. The speaker uses colored dots to demonstrate that what we see can differ based on how our brains interpret visual information, influenced by context.

Why does the speaker claim that 'context is everything' when it comes to perception?

The speaker argues that context shapes how we interpret sensory information. This is because the light reaching our eyes is influenced by various factors like the object's color, the light source, and the surrounding environment. Our brain uses context to make sense of this ambiguous information.

What does the example of the predator in the jungle scene demonstrate?

It shows that color perception is crucial for survival. Without being able to differentiate colors based on context, we might not notice a camouflaged predator in time, demonstrating how vital accurate perception is in recognizing threats.

How does the speaker explain the concept that 'sensory information is meaningless'?

The speaker suggests that sensory information, such as the light that falls onto our eyes, could be interpreted in countless ways because it depends on multiple factors. It is the brain's interpretation, based on experience and context, that gives this information meaning.

What does the exercise involving staring at the dot between the green and red squares illustrate?

This exercise demonstrates how our brains adapt to new contexts. After staring at the dot, the brain assumes different lighting conditions for each side, altering our perception of the identical desert scenes when viewed side by side, making them appear different.

How does the experiment with the identical tiles in different contexts relate to perception?

The experiment shows that our perception of an object’s brightness or color can change depending on its surrounding context. Identical tiles appear different because the brain interprets their colors based on the assumed lighting and shadow conditions around them.

What does the example with the spinning diamond illustrate about perception?

It illustrates perceptual ambiguity and how our brain can switch between different interpretations. The diamond appears to spin in opposite directions depending on our focus or actions like blinking, demonstrating that our perception can change even with the same visual input.

Why does the speaker discuss illusions experienced by bumblebees?

The speaker highlights that even simple organisms like bumblebees experience illusions and interpret visual information based on context. This suggests that the ability to perceive the world through contextual relationships is fundamental across species.

What is the significance of translating visual information into sound, as described in the experiment with David?

Translating visual information into sound helps people with visual impairments navigate their environment. It also provides insight into how the brain processes and interprets sensory information, showing that perception is flexible and can adapt to new sensory inputs.

What is the main takeaway the speaker wants the audience to remember?

The speaker wants the audience to understand that perception is not about seeing the world as it is, but as it was useful to see in the past. Our brains constantly redefine what is 'normal' based on experience and context, which affects how we perceive and interact with the world.

Outlines

00:00

🎮 Game of Perception: Can You See the Reality?

The speaker introduces a game where the audience must identify matching dots between two panels. Most fail, revealing how context affects perception. The speaker discusses how nearly every living organism has evolved to detect light and colors, yet even this fundamental ability is heavily influenced by context. The broader discussion begins by questioning not only what color is but why context shapes perception and influences our understanding of reality.

05:01

👀 The Purpose of Color in Survival

Through a jungle scene, the speaker demonstrates how color helps detect threats, like a predator. The brain deciphers colors based on multiple factors, including the color of the object, illumination, and the medium between the observer and the object. Perception becomes complex because the same sensory input could come from vastly different real-world sources. The speaker emphasizes that sensory information is meaningless on its own—it’s our brain that gives it significance through learned associations.

10:02

🧠 How the Brain Redefines Normality

The speaker shows how quickly the brain can redefine normality using an example of two desert scenes. The brain adjusts to different light sources and interprets identical images differently based on perceived light conditions. The speaker illustrates how context impacts perception, using the example of identical tiles in different light surrounds. The brain’s interpretation of these tiles varies based on learned behaviors and past experiences, leading to powerful illusions.

15:03

🎨 Creating Powerful Visual Illusions

Using more complex illusions, the speaker explores how our brains interpret color and light. Identical tiles in different surroundings can appear completely different due to the brain’s learned behavior of associating context with light. The speaker introduces a memorable illusion where seemingly different colored tiles are revealed to be the same, challenging the audience’s perception and reinforcing the idea that context drives how we interpret the world.

🐝 Bumblebee Perception and Human Illusions

The speaker shifts to the study of bumblebees, showing that even bees experience illusions. In a controlled environment, bees can learn through their experiences, solving puzzles and perceiving colors similarly to humans. This reveals that sensory perception is not fragile but rather an evolved mechanism for survival, shaped by past usefulness. The speaker reinforces that the brain evolved not to see the world as it is but as it was useful to see in the past.

🎵 Translating Visuals into Sound: The Power of Brain Plasticity

The speaker discusses the brain’s plasticity by showcasing a project where light is translated into sound, enabling people to navigate the world using auditory cues. A blindfolded individual locates a plate based on sound alone, showing how perception adapts. The experiment extends to creating music from color, highlighting how children can compose orchestral pieces by translating visual patterns into sound. This demonstrates the brain’s adaptability and the interconnectedness of senses.

🌍 We Are Products of Our Environment

The speaker emphasizes that no one is an outside observer of nature; we are deeply influenced by our environment and interactions with it. Our perception is relative, shaped by historical and empirical experiences. The final demonstration uses colored filters on surfaces, challenging the audience’s certainty about what they perceive. The conclusion encourages the audience to embrace uncertainty as a path to deeper understanding, leaving them with a thought-provoking question about whether what they see is truly an illusion.

Keywords

💡Color Perception

Color perception refers to how our brain interprets and differentiates colors based on light that reaches our eyes. In the video, the speaker demonstrates how our perception of color is influenced by context, such as lighting and surroundings, showing that identical colors can appear different depending on their context. This illustrates that our perception is not a direct reflection of reality but is shaped by our brain's interpretation of sensory information.

💡Context

Context is the surrounding conditions or factors that affect how we perceive sensory information. The video emphasizes that context is crucial in determining how we see and interpret the world, using examples like colored tiles and illusions to show that identical sensory inputs can lead to different perceptions based on context. This idea challenges the notion of objective reality, suggesting that our understanding of the world is relative to the context in which we experience it.

💡Illusion

An illusion is a misleading or deceptive perception of reality. In the video, illusions are used to show that our senses can be tricked into seeing things differently from how they physically are. For example, the speaker uses color and light to create illusions where identical colors appear different, demonstrating that our brain constructs reality based on prior experiences and context rather than direct sensory input.

💡Sensory Information

Sensory information is the data received by our senses from the external environment. The video explains that sensory information, such as light entering our eyes, is inherently meaningless because it can correspond to multiple real-world sources. The brain interprets this information by finding patterns and assigning meaning based on past experiences, illustrating that perception is an active process of constructing reality.

💡Learning to See

Learning to see refers to the process by which the brain develops the ability to interpret sensory information through experience. The video discusses how our perception is shaped by learning patterns and associations from our interactions with the world. This concept is exemplified when the speaker shows how our brains can quickly adapt to new contexts, such as adjusting to different lighting conditions in the desert scene.

💡Retinal Information

Retinal information is the visual data captured by the retina, the light-sensitive layer at the back of the eye. The video highlights that this information is not sufficient on its own to create a coherent picture of reality because it is influenced by various factors like lighting and viewing angle. The brain must interpret and reconstruct this information to create a meaningful perception, emphasizing the active role of the brain in vision.

💡Behavioral Meaning

Behavioral meaning is the significance that the brain assigns to sensory information based on its relevance to past experiences and potential actions. The speaker explains that the brain evolved to associate certain patterns of sensory input with appropriate behaviors, which helps us navigate and interact with our environment. For example, recognizing a predator in the jungle based on subtle visual cues is a survival mechanism that relies on assigning behavioral meaning to sensory data.

💡Plasticity

Plasticity refers to the brain's ability to adapt and change in response to new experiences. The video illustrates this concept by showing how quickly our perception of normality can change, such as when the brain adapts to different lighting conditions and reinterprets the color of desert scenes. This flexibility is fundamental to how we learn and perceive the world, allowing us to continuously redefine what is normal.

💡Relative Perception

Relative perception is the idea that our perception of sensory information is influenced by comparisons and contrasts with surrounding elements rather than absolute values. The speaker demonstrates this by showing how two identical colors can appear different depending on their background, illustrating that perception is not based on the objective properties of objects but on their relative context.

💡Ecology of Perception

Ecology of perception refers to the idea that our perceptions are shaped by the interactions between our sensory systems and the environment. The video suggests that our perception is not just a passive reception of sensory data but an active process influenced by our surroundings and experiences. This concept challenges the traditional view of perception as a direct and objective process, highlighting its dynamic and context-dependent nature.

Highlights

The game challenges the audience to identify matching colored dots between two panels, emphasizing how context affects perception.

Even basic visual tasks, like color perception, are influenced by the brain’s interpretation of context rather than objective reality.

The speaker demonstrates how color helps us distinguish between objects in the environment, such as detecting a predator in a jungle scene.

The brain processes light, not just based on the color of objects, but also considering the color of illumination and the space between the viewer and the object.

Sensory information is inherently meaningless until the brain assigns behavioral meaning to it based on past experiences.

The speaker highlights that the brain evolved to recognize patterns and relationships in sensory data, shaping how we interpret information.

Contextual perception can redefine reality, demonstrated by the brain seeing two identical desert scenes differently after adapting to red and green light.

Identical tiles can appear different depending on the surrounding light and shadows, influenced by the brain’s learned expectations from past experiences.

Perception of motion is subjective and can be manipulated by the brain, as demonstrated with a spinning diamond illusion that changes direction based on focus.

Even bumblebees experience visual illusions and solve problems by using relationships between colors to navigate complex environments.

The brain doesn’t aim to perceive the world as it is, but rather as it was useful to see in the past for survival and interaction.

By translating light into sound, visually impaired individuals can ‘hear’ their environment and navigate using sound-based perception.

A six-year-old child composed a piece of music for a 32-piece orchestra by translating colors into sound, illustrating the brain’s plasticity in interpreting sensory input.

The speaker argues that embracing uncertainty can lead to a better understanding of perception and our interaction with the world.

The final demonstration shows two sets of tiles under different lighting conditions, leading the audience to question whether their differing appearance is an illusion.

Transcripts

I want to start with a game. Okay?

And to win this game,

all you have to do is see the reality that's in front of you

as it really is, all right?

So we have two panels here, of colored dots.

And one of those dots is the same in the two panels.

And you have to tell me which one.

Now, I narrowed it down

to the gray one, the green one, and, say, the orange one.

So by a show of hands, we'll start with the easiest one.

Show of hands: how many people think it's the gray one?

Really? Okay.

How many people think it's the green one?

And how many people think it's the orange one?

Pretty even split.

Let's find out what the reality is.

Here is the orange one.

(Laughter)

Here is the green one.

And here is the gray one.

(Laughter)

So for all of you who saw that, you're complete realists.

All right?

(Laughter)

So this is pretty amazing, isn't it?

Because nearly every living system

has evolved the ability to detect light in one way or another.

So for us, seeing color

is one of the simplest things the brain does.

And yet, even at this most fundamental level,

context is everything.

What I'm going to talk about is not that context is everything,

but why context is everything.

Because it's answering that question

that tells us not only why we see what we do,

but who we are as individuals,

and who we are as a society.

But first, we have to ask another question,

which is, "What is color for?"

And instead of telling you, I'll just show you.

What you see here is a jungle scene,

and you see the surfaces according to the amount of light

that those surfaces reflect.

Now, can any of you see the predator that's about to jump out at you?

And if you haven't seen it yet, you're dead, right?

(Laughter)

Can anyone see it? Anyone? No?

Now let's see the surfaces

according to the quality of light that they reflect.

And now you see it.

So, color enables us to see

the similarities and differences between surfaces,

according to the full spectrum of light that they reflect.

But what you've just done

is in many respects mathematically impossible.

Why?

Because, as Berkeley tells us,

we have no direct access to our physical world,

other than through our senses.

And the light that falls onto our eyes

is determined by multiple things in the world,

not only the color of objects,

but also the color of their illumination,

and the color of the space between us and those objects.

You vary any one of those parameters,

and you'll change the color of the light that falls onto your eye.

This is a huge problem,

because it means that the same image

could have an infinite number of possible real-world sources.

Let me show you what I mean.

Imagine that this is the back of your eye, okay?

And these are two projections from the world.

They're identical in every single way.

Identical in shape, size, spectral content.

They are the same, as far as your eye is concerned.

And yet they come from completely different sources.

The one on the right comes from a yellow surface,

in shadow, oriented facing the left,

viewed through a pinkish medium.

The one on the left comes from an orange surface,

under direct light, facing to the right,

viewed through sort of a bluish medium.

Completely different meanings,

giving rise to the exact same retinal information.

And yet it's only the retinal information that we get.

So how on Earth do we even see?

So if you remember anything in this next 18 minutes,

remember this:

that the light that falls onto your eye,

sensory information, is meaningless,

because it could mean literally anything.

And what's true for sensory information is true for information generally.

There's no inherent meaning in information.

It's what we do with that information that matters.

So, how do we see? Well, we see by learning to see.

The brain evolved the mechanisms for finding patterns,

finding relationships in information,

and associating those relationships with a behavioral meaning,

a significance, by interacting with the world.

We're very aware of this

in the form of more cognitive attributes, like language.

I'm going to give you some letter strings,

and I want you to read them out for me, if you can.

Audience: "Can you read this?"

"You are not reading this."

"What are you reading?"

Beau Lotto: "What are you reading?" Half the letters are missing, right?

There's no a priori reason

why an "H" has to go between that "W" and "A."

But you put one there. Why?

Because in the statistics of your past experience,

it would have been useful to do so.

So you do so again.

And yet you don't put a letter after that first "T."

Why? Because it wouldn't have been useful in the past.

So you don't do it again.

So, let me show you how quickly our brains can redefine normality,

even at the simplest thing the brain does, which is color.

So if I could have the lights down up here.

I want you to first notice that those two desert scenes are physically the same.

One is simply the flipping of the other.

Now I want you to look at that dot

between the green and the red.

And I want you to stare at that dot. Don't look anywhere else.

We're going to look at it for about 30 seconds,

which is a bit of a killer in an 18-minute talk.

(Laughter)

But I really want you to learn.

And I'll tell you -- don't look anywhere else --

I'll tell you what's happening in your head.

Your brain is learning,

and it's learning that the right side of its visual field

is under red illumination;

the left side of its visual field is under green illumination.

That's what it's learning. Okay?

Now, when I tell you, I want you to look at the dot between the two desert scenes.

So why don't you do that now?

(Laughter)

Can I have the lights up again?

I take it from your response they don't look the same anymore, right?

(Applause)

Why? Because your brain is seeing that same information

as if the right one is still under red light,

and the left one is still under green light.

That's your new normal.

Okay? So, what does this mean for context?

It means I can take two identical squares,

put them in light and dark surrounds,

and the one on the dark surround looks lighter than on the light surround.

What's significant is not simply the light and dark surrounds that matter.

It's what those light and dark surrounds meant for your behavior in the past.

So I'll show you what I mean.

Here we have that exact same illusion.

We have two identical tiles on the left,

one in a dark surround, one in a light surround.

And the same thing over on the right.

Now, I'll reveal those two scenes,

but I'm not going to change anything within those boxes,

except their meaning.

And see what happens to your perception.

Notice that on the left

the two tiles look nearly completely opposite:

one very white and one very dark, right?

Whereas on the right, the two tiles look nearly the same.

And yet there is still one on a dark surround,

and one on a light surround.

Why?

Because if the tile in that shadow were in fact in shadow,

and reflecting the same amount of light to your eye

as the one outside the shadow,

it would have to be more reflective -- just the laws of physics.

So you see it that way.

Whereas on the right, the information is consistent

with those two tiles being under the same light.

If they're under the same light reflecting the same amount of light to your eye,

then they must be equally reflective.

So you see it that way.

Which means we can bring all this information together

to create some incredibly strong illusions.

This is one I made a few years ago.

And you'll notice you see a dark brown tile at the top,

and a bright orange tile at the side.

That is your perceptual reality.

The physical reality is that those two tiles are the same.

Here you see four gray tiles on your left,

seven gray tiles on the right.

I'm not going to change those tiles at all,

but I'm going to reveal the rest of the scene.

And see what happens to your perception.

The four blue tiles on the left are gray.

The seven yellow tiles on the right are also gray.

They are the same. Okay?

Don't believe me? Let's watch it again.

What's true for color is also true for complex perceptions of motion.

So, here we have --

let's turn this around -- a diamond.

And what I'm going to do is, I'm going to hold it here,

and I'm going to spin it.

And for all of you, you'll see it probably spinning this direction.

Now I want you to keep looking at it.

Move your eyes around, blink, maybe close one eye.

And suddenly it will flip, and start spinning the opposite direction.

Yes? Raise your hand if you got that. Yes?

Keep blinking.

Every time you blink, it will switch.

So I can ask you, which direction is it rotating?

How do you know?

Your brain doesn't know, because both are equally likely.

So depending on where it looks,

it flips between the two possibilities.

Are we the only ones that see illusions?

The answer to this question is no.

Even the beautiful bumblebee,

with its mere one million brain cells,

which is 250 times fewer cells than you have in one retina,

sees illusions, does the most complicated things

that even our most sophisticated computers can't do.

So in my lab we work on bumblebees,

because we can completely control their experience,

and see how it alters the architecture of their brain.

We do this in what we call the Bee Matrix.

Here you have the hive.

You can see the queen bee, the large bee in the middle.

Those are her daughters, the eggs.

They go back and forth between this hive and the arena, via this tube.

You'll see one of the bees come out here.

You see how she has a little number on her?

There's another one coming out, she also has a number on her.

Now, they're not born that way, right?

We pull them out, put them in the fridge, and they fall asleep.

Then you can superglue little numbers on them.

(Laughter)

And now, in this experiment they get a reward if they go to the blue flowers.

They land on the flower,

stick their tongue in there, called a proboscis, and drink sugar water.

She's drinking a glass of water that's about that big to you and I,

will do that about three times, then fly.

And sometimes they learn not to go to the blue,

but to go where the other bees go.

So they copy each other.

They can count to five. They can recognize faces.

And here she comes down the ladder.

And she'll come into the hive, find an empty honey pot,

and throw up, and that's honey.

(Laughter)

Now remember, she's supposed to be going to the blue flowers,

but what are these bees doing in the upper right corner?

It looks like they're going to green flowers.

Now, are they getting it wrong?

And the answer to the question is no. Those are actually blue flowers.

But those are blue flowers under green light.

So they're using the relationships between the colors to solve the puzzle,

which is exactly what we do.

So, illusions are often used,

especially in art, in the words of a more contemporary artist,

"to demonstrate the fragility of our senses."

Okay, this is complete rubbish.

The senses aren't fragile. And if they were, we wouldn't be here.

Instead, color tells us something completely different,

that the brain didn't actually evolve to see the world the way it is.

We can't.

Instead, the brain evolved to see the world

the way it was useful to see in the past.

And how we see is by continually redefining normality.

So, how can we take this incredible capacity of plasticity of the brain

and get people to experience their world differently?

Well, one of the ways we do it in my lab and studio

is we translate the light into sound,

and we enable people to hear their visual world.

And they can navigate the world using their ears.

Here's David on the right, and he's holding a camera.

On the left is what his camera sees.

And you'll see there's a faint line going across that image.

That line is broken up into 32 squares.

In each square, we calculate the average color.

And then we just simply translate that into sound.

And now he's going to turn around,

close his eyes,

and find a plate on the ground with his eyes closed.

(Continuous sound)

(Sound changes momentarily)

(Sound changes momentarily)

(Sound changes momentarily)

(Sound changes momentarily)

(Sound changes momentarily)

Beau Lotto: He finds it. Amazing, right?

So not only can we create a prosthetic for the visually impaired,

but we can also investigate

how people literally make sense of the world.

But we can also do something else.

We can also make music with color.

So, working with kids,

they created images,

thinking about what might the images you see

sound like if we could listen to them.

And then we translated these images.

And this is one of those images.

And this is a six-year-old child composing a piece of music

for a 32-piece orchestra.

And this is what it sounds like.

(Electronic representation of orchestral music)

So, a six-year-old child. Okay?

Now, what does all this mean?

What this suggests is that no one is an outside observer of nature, okay?

We're not defined by our central properties,

by the bits that make us up.

We're defined by our environment and our interaction with that environment,

by our ecology.

And that ecology is necessarily relative, historical and empirical.

So, what I'd like to finish with is this over here.

Because what I've been trying to do is really celebrate uncertainty.

Because I think only through uncertainty is there potential for understanding.

So, if some of you are still feeling a bit too certain,

I'd like to do this one.

So, if we have the lights down.

And what we have here --

Can everyone see 25 purple surfaces on your left,

and 25, call it yellowish, surfaces on your right?

So now, what I want to do,

I'm going to put the middle nine surfaces here

under yellow illumination,

by simply putting a filter behind them.

Now you can see that changes the light that's coming through there, right?

Because now the light is going through a yellowish filter

and then a purplish filter.

I'm going to do the opposite on the left here.

I'm going to put the middle nine under a purplish light.

Now, some of you will have noticed that the consequence

is that the light coming through those middle nine on the right,

or your left, is exactly the same as the light

coming through the middle nine on your right.

Agreed? Yes?

Okay. So they are physically the same.

Let's pull the covers off.

Now remember --

you know that the middle nine are exactly the same.

Do they look the same?

No.

The question is, "Is that an illusion?"

And I'll leave you with that.

So, thank you very much.

(Laughter)

(Applause)