Digital semiotics: making sense of the world | Oscar Bastiaens | TEDxDordrecht

Summary

The speaker argues against the overemphasis on big data, advocating for a deeper understanding of semiotics to interpret data meaningfully. They discuss creating immersive virtual experiences, like the 'Kinetic' film, to explore the human response to virtual touch. The talk delves into the importance of recognizing the meta-narrative in digital media and the need for a digital semiotic system to fully comprehend the complex data landscape, suggesting we should focus on understanding how data communicates with us rather than just learning to code.

Takeaways

• 📊 Big data is seen as a quick win rather than a meaningful resource by many digital media producers.
• 🔍 The speaker is on a mission to enlighten people about using data in more meaningful ways, emphasizing the importance of semiotics in understanding data.
• 🌐 The participatory online world and our physical reality are becoming increasingly intertwined, raising questions about the nature of reality and our perception of it.
• 🎓 The speaker works with students and researchers to create virtual worlds that can transport consciousness and provide realistic experiences.
• 🎬 The Ulus Rift film 'This Kinetic' allows users to determine their perspective, offering a 360-degree range of choices, showcasing the potential of virtual reality.
• 🤔 The speaker explores the dilemma of decision-making in a virtual coma experience, where users feel they are in a coma and react to virtual stimuli.
• 🧠 Semiotics, the study of signs and meaning-making, is crucial for understanding the super system of science that is expanding with digital and virtual data.
• 🌐 The concept of a parallel virtual world has been explored since the 1950s, leading to questions about the nature of reality and our ability to discern it.
• 🤝 The speaker advocates for the creation of a digital semiotic system to understand the meta narrative of data and improve digital services.
• 📚 Teaching digital semiotics is proposed as a way to help people understand how the digital system communicates with us and make sense of the data around us.

Q & A

What is the speaker's main contention about big data?

The speaker contends that big data is currently seen as everything, but they argue it is actually nothing because it is often used for quick wins rather than creating meaningful digital experiences.

What is the speaker's crusade in the digital industry?

The speaker's crusade is to enlighten people about using data in a more meaningful way and to understand the value of online data as part of a 'super system of signs'.

Why does the speaker believe we should study semiotics?

The speaker believes we should study semiotics to understand how the digital and virtual environment communicates with us in a meaningful way, similar to how Google finishes our search queries.

What is the significance of the 'Ulus Rift film' mentioned in the script?

The 'Ulus Rift film' is significant because it demonstrates the speaker's work in creating immersive virtual experiences where users determine their perspective and view within a 360-degree range of choices.

What was the narrative purpose of the coma experience in the film 'this kinetic'?

The narrative purpose of the coma experience in the film 'this kinetic' was to immerse the audience into the perspective of a coma patient, allowing them to hear and see their surroundings as part of the story.

What is semiotics and why is it important according to the speaker?

Semiotics is the study of meaning-making, stemming from linguistics and expanding into symbols, gestures, and expressions of man and animal behavior. It is important because it helps us understand the meta narrative of data and the digital and virtual environment.

What are the three possibilities Nick Bostrom presents in his simulation argument?

Nick Bostrom presents three possibilities: 1) We will never reach the stage where technology can simulate a world that looks and feels as real as ours, 2) We will reach that advanced technology but never use it due to potential dangers, or 3) If the first two options are proven false, then we are most likely already living in a virtual world.

What is the concept of 'simulakra' as described by Jean Baudrillard?

According to Jean Baudrillard, 'simulakra' are copies of something that never existed, where the truth is concealed by the fact that there is none. It suggests that we have begun to replace meaning with signs and symbols, leading to a simulation of reality.

Why are many online services failing to cater to our needs of interpretation with the digital system?

Many online services are failing to cater to our needs of interpretation because they have ceased to understand the meta narrative of data and the connections that can be made within the digital system.

What is the speaker's proposed solution to improve digital systems' understanding of user needs?

The speaker proposes that we need a digital semiotic system that can understand the full meta narrative of data, including the ability to read text, visuals, and user preferences to create a valuable discourse with users.

Outlines

00:00

📊 The Misuse of Big Data and the Need for Semiotics

The speaker challenges the notion that big data is everything, arguing that it's often used for quick wins rather than meaningful experiences. They express a commitment to enlightening people about the potential for data to be used more meaningfully, suggesting that the digital and physical worlds are becoming increasingly intertwined. The speaker emphasizes the importance of semiotics, the study of signs and symbols, as a way to understand and communicate with the complex system of data we are creating online. They share their work with students and researchers to create virtual worlds that can transport consciousness and provide realistic experiences, using the example of a film called 'This Kinetic' to illustrate how virtual reality can offer a 360-degree range of choices and perspectives. The speaker concludes by advocating for the study of semiotics to better understand the meaning and value of our digital interactions.

05:01

🌐 Exploring the Concept of Parallel Worlds and Reality in Virtual Reality

The speaker delves into the philosophical implications of virtual reality, questioning what constitutes 'real' and whether we need a physical reality to experience something that feels real. They introduce two theories related to virtual reality: Nick Bostrom's simulation argument, which posits that we might already be living in a simulated world, and Jean Baudrillard's concept of simulacra, which suggests that our experiences are becoming simulations of reality, replacing meaning with signs and symbols. The speaker uses the example of a theater performance and its promotion on Facebook to illustrate how digital systems fail to understand the meta-narrative of user experiences and interactions, advocating for a digital semiotic system that can comprehend and respond to the dynamic nature of user data and preferences.

10:01

💡 The Importance of Digital Semiotics in Understanding User Data

The speaker argues for the necessity of a digital semiotic system to fully understand the meta-narrative of data and user interactions. They emphasize the dynamic nature of online data and text, which can change based on user desires, and the limitations of systems that only respond to specific input characteristics. The speaker calls for a shift in focus from teaching people how to code to teaching them how to understand the super system of science that is data, and how it communicates with us. They conclude by urging the study and understanding of digital semiotics to grasp the larger narrative of data and to create more meaningful and valuable digital interactions.

Keywords

💡Big Data

Big Data refers to the vast amounts of structured and unstructured data that are so large they become difficult to process using traditional database management tools. In the context of the video, the speaker criticizes the misuse of big data for quick wins rather than for creating meaningful digital experiences. The speaker's contention is that big data is 'nothing' if not used to question its value and to enhance understanding through semiotics.

💡Semiotics

Semiotics is the study of signs and symbols and their use or interpretation. It is a field that originated from linguistics but has expanded to include various forms of communication. The speaker emphasizes the importance of semiotics in understanding the 'super system of signs' in digital and virtual environments. The video suggests that to truly grasp the meaning of data, we must study semiotics to decode the signs and symbols within our digital experiences.

💡Digital Experiences

Digital experiences are interactive, engaging, and immersive environments or applications accessed through digital devices. The video discusses how digital media producers aim to create enhanced digital experiences but often fall short due to a lack of understanding of the underlying semiotics. The speaker's goal is to use data in a way that enriches these experiences, making them more meaningful and interactive.

💡Virtual Reality (VR)

Virtual Reality is a computer-generated simulation of a three-dimensional environment that can be interacted with in a seemingly real or physical way by a person using special electronic equipment, such as a helmet with a screen inside or gloves fitted with sensors. The video references VR as a tool for understanding the complexity of data and as a medium that can transport consciousness into different worlds, creating highly realistic experiences.

💡Meta Narrative

A meta narrative is a grand or overarching story that provides a context for understanding specific events or stories. In the video, the speaker discusses the need to understand the meta narrative of data to create satisfying media products and digital applications. The meta narrative is depicted as a framework that connects various media products and digital services, guiding their ethical values and user interactions.

💡Simulacrum

A simulacrum is a representation or imitation of something, often to the point where it is indistinguishable from the original. The video references Jean Baudrillard's concept of simulacra, where signs and symbols replace the actual experience of reality, leading to a simulation of reality. The speaker uses this concept to critique the current state of digital media, which often fails to provide a meaningful interpretation of data.

💡Data Interpretation

Data interpretation involves analyzing data to draw conclusions or make predictions. The video argues that many online services are failing to cater to users' needs for data interpretation, as they are focused on quick wins rather than understanding the meta narrative of the data. The speaker suggests that a deeper understanding of semiotics is necessary to properly interpret and utilize data.

💡Oculus Rift

The Oculus Rift is a virtual reality headset that allows users to experience immersive, interactive environments. The video mentions a film made using the Oculus Rift, where the audience is immersed in a narrative experience of being in a coma. This example illustrates how VR can be used to create highly engaging and emotionally resonant digital experiences.

💡Digital Semiotic System

A digital semiotic system refers to a framework or methodology for understanding and interpreting signs, symbols, and communication within digital environments. The speaker advocates for the development of such a system to better understand the meta narrative of data and to create more meaningful digital interactions. The system would need to be dynamic and responsive to the changing nature of user-generated content.

💡User-Generated Content

User-generated content is any form of content, such as text, photos, videos, or audio, that is created by users and published on the internet. The video discusses how all the data and text created online by users is dynamic and can change at any time. This dynamic nature presents a challenge for digital systems that need to understand and respond to user-generated content in a meaningful way.

💡Quick Win

A quick win refers to a short-term gain or achievement that provides immediate benefits but may not have long-term value or sustainability. In the video, the speaker criticizes the use of data for quick wins, suggesting that a more meaningful approach is needed. The term is used to highlight the short-sightedness of some digital media strategies that prioritize immediate gains over deeper understanding and long-term value.

Highlights

The speaker argues that big data is not everything and that its current use is often for quick wins rather than meaningful experiences.

The speaker's crusade is to enlighten people on using data in a more meaningful way.

The importance of understanding the participatory world and its intertwining with the online world is emphasized.

The speaker advocates for studying semiotics to understand how data can communicate with us meaningfully.

The speaker works with students and researchers to create virtual worlds that can transport consciousness.

The 'Kinetic' film project is mentioned, where users determine perspective in a 360-degree range.

The 'Ulus Rift' film premiered at the Go Short International Short Film Festival, immersing the audience in a coma narrative.

The dilemma of decision-making in a virtual coma experience is explored.

The speaker is fascinated by how people create meaning from virtual experiences.

Semiotics is introduced as the study of signs and meaning-making, essential for understanding digital and virtual signification.

The speaker discusses the 'super system of science' as a vast database of signs expanding with user contributions.

The concern of distinguishing real from virtual experiences is raised, with references to theories by Nick Bostrum.

Jean Baudrillard's concept of simulacra is introduced, where signs and symbols replace meaning.

The meta narrative of media products and digital media is explained, highlighting the need for understanding shared ethical values.

A real-life example of Facebook's algorithm failing to understand user dissatisfaction with a theater performance is given.

The necessity for a digital semiotic system to understand the full meta narrative of data is argued.

The speaker calls for teaching digital semiotics to understand the meta narrative of data, rather than just teaching coding.

Transcripts

many people think big data is everything

well I contend it is actually

nothing many digital media producers

nowadays are trying to create enhanced

and meaningful digital

experiences and all the data gathered

online and produced online is now just

used for the quick win

very few people in the industry actually

question the value of what we're doing

with data and in the past few months it

has become my crusade to Enlighten

people about how we can use data in a

more meaningful

way how we can use this online data this

super system of signs in a more

meaningful way is my

goal I'd like to give you a peek into my

everyday thoughts because I believe that

this fast moving participatory world we

are currently living in and the one we

are creating online are getting

intertwined even more so I believe that

if we do not stop worrying about data

and start studying and understanding

semiotics we could be pinching ourselves

until our arms turn blue and even then

we'd have no clue of whether what we

experience is real or

not I'd like all of us to understand the

bigger picture of data and not by

learning how to code but learning how to

understand how this system this super

system of science can communicate with

us in a more meaningful way just like

Google already does finishing our search

questions daily I work together with

students and fellow researchers in

creating and understanding Virtual

Worlds that are able to transport our

Consciousness into a different

world creating an experience that feels

so

real that even a virtual touch feels

like a real

one and in that respect I'd like to tell

you something about the Ulus Rift film

that we made last year called this

kinetic a film in which the user

determines what perspective and what

view is going to be looked at offering

them a 360 Dee range of choices of

options this is a picture from go short

International short film festival in NAA

and we had our premiere for our short

film right here we immersed our audience

into experiencing a

coma for narrative purposes of course we

made sure that you as the coma patient

were able to hear and see your

surroundings all interwoven with the

story when people stepped into our

ambulance and wore the Oculus under

heads together with a headphone they

stepped into this dark world a world in

which your family is standing around

your bedside wondering what to do with

you in an eternal

coma the Dilemma of decision- making is

being played out and you are playing the

lead

role at a certain point in the film your

sister touches your leg now I've

observed our so-called patients and

found that many of them move their leg

at the moment of virtual touch just

quickly ensuring for themselves that all

of this is not

real I still am fascinated by how we

make such understanding and create

meaning from these Virtual Worlds and I

came to the conclusion that for myself

to understand all of this I needed to

study semiotics and people often ask me

what is

semiotics what are you actually

researching and the answer to be honest

I don't know

yet but luckily there's one large

General agreement on what semiotics is

it is the signs of signs the study of

meaning making stemming from Linguistics

it has now grown into the fields of

symbols gestures an expression of man

and animal

behavior and now it has found its way

into the field of digital and virtual

signification and it are these two new

fields that have a vast majority of

unknown signs and symbols yet to Define

and

identify the title of my talk refers to

the super system of science the

humongous datab base of letters words

images shapes objects that is expanding

with every new entry made by you and me

users of the digital and virtual

environment and in order not to leave

you all too confused at the end of my

talk we'll take one leading example in

our journey to understand why we need

semiotics for that we'll use Virtual

Reality as a means to understanding this

complexity because I believe that

virtual reality is something we all have

some sort of an idea with from films

like Tron and Matrix we've already

played with the idea of a possible

second virtual world existing besides

our own already since the 50s mankind

has played around with the idea of a

parallel

world now the interesting thing here is

that this thinking of a parallel world

brings us to the concern many people

have we do not need a reality to

experience something that feels

real but then what is real can I still

trust myself in identifying the real

world from a virtual one that might look

and feel exactly the

same and there are two very interesting

theories that have been developed over

time when it comes to virtual reality

that I'd like to share and the first

scholar I'd like to mention here is Nick

bustr he claimed in this simulation

argument that there is one truth when it

comes to virtual reality that time will

have to show out in totally he says

there are three

possibilities either we will never reach

the stage age in which technology can

simulate a world that looks and feels

just as real as this

one or second he says we will reach that

advanced technology but we will never

use it because of the dangers it might

hold or third when the two former

options are both proven false in time

then we are most definitely most

definitely already in a virtual world

right

now the second scholar I'd like to

mention here is Jean

he's the founder of the notion simulakra

and similation describing the simulakra

as never that which conceals the truth

it is the truth which conceals that

there is none the simulacra therefore is

true so I'll explain this a little bit

according to bodar we've already started

to replace meaning with signs and

symbols resulting in The Human

Experience being a similation of

reality let me give you an example we

can all imagine how it would be like to

sit inside a crashing plane even though

we all never had the actual

experience this simulation in symbols

and signs has made us believe that this

is

truth what has happened is that we've

ceased to understand the larger picture

of this all the larger picture of

data but what went wrong in this fight

for data why are so many online services

failing to cater to our needs of

interpretation with the digital system

and that is because we've ceased to

understand the meta Narrative of it all

and The Meta narrative looks a bit like

this and let me explain the meta

narrative and this model behind me and

maybe best for that is if I use media

products and ethics something we're all

familiar with so imagine all these small

blogs being a media product a television

series series a book a game a film

because there are many media producers

out there and all of them create

different stories on different

platforms but all of them have an

understanding of the general agreement

on specific shared ethical values a meta

narrative without them understanding

this meta

narrative they're unable to create

satisfying media products for us the

audience

another example is digital media because

the same goes for digital media imagine

those blocks not being media products

but imagine them being applications you

use on your phone or your

tablet many of them are using the data

now just for a quick win unable to

understand our communication with the

system unable to understand the meta

narrative and the connections that can

be

made an example of this last week I went

to see a a theater performance with a

friend of mine and we did not really

know what to expect but we had some ey

hopes and my friend took a picture of

the performances poster posted it on

Facebook and said looking

forward now immediately the Facebook

algorithm went to search for similar

performances that they could advertise

to

him however when after the First Act we

found that we did not like the

performance at all and left actually he

commented on his Facebook that he did

not like it now what should the

algorithm read at this point and I think

there are two very important steps that

can be taken in

between first is that we have to make

sure that we create a system that is

able to not only read the text on the

POR but also the

visuals and therefore it is able to

dissect genre

style place of performance and Link that

to the other

data second is that we have to create a

system which can understand that this

person didn't not like the performance

at all and realizing as a system that it

might be smart not to advertise similar

performances

anymore you see these are a few of the

reasons why I think that we are in need

of a digital semiotic system to really

get a hold and an understanding of this

meta narrative to its full extent

something we still have a journey to

walk to

because all the data and text we are

creating online is

dynamic and that means that it can

change any time into anything the user

desires if I perform a Google search I

can get text to become image all of a

sudden or even audio visual

content and the problem will be if that

I create an system that is only

responding to specific input

characteristics if it can only respond

to text

it doesn't know what to do with my image

it will never completely understand me

as a user let alone being able to create

something far more important than just

that which is a valuable discourse with

me the

user see the problem is and that is also

my crusade is that we're now teaching

people just to use data we're teaching

them how to code and instead I believe

that we should teach them an

understanding of how this super system

of science is able to talk to us make

sense for us the

users I think it is time that we teach

and understand and study semiotics

digital semiotics and understand the

meta Narrative of data thank

you