The creative interface: connecting art and computer science | Cole Wiley | TEDxLSU

Summary

The speaker, with a background in sculpture and computer science, shares their journey of blending these fields to create new media art and interactive installations. They discuss their first project, 'Riverless Walk,' which allowed exploration of the Chicago Riverwalk through various mediums, integrating traditional art forms with technology. The speaker's work in human-computer interaction and natural user interfaces led to a software job at a startup, Kenobi, developing an innovative app for real-time kinesthetic activity feedback. They encourage finding connections between unrelated interests, as these can lead to unexpected and rewarding outcomes.

Takeaways

• 🎨 The speaker has a background in both sculpture and computer science, and found success by integrating these fields.
• 🤝 Professors played a crucial role in encouraging the blending of computer science and art, leading to the creation of new media art and interactive installations.
• 🚀 The speaker's first interactive piece, 'Allies', was a pivotal project that taught them more about creating a minimum viable product and software than any computer science class.
• 💡 A 'eureka moment' during a critique session led to the realization that the practices in software development, like the two-week agile development cycle, mirrored those in art.
• 🗣 The speaker emphasizes that 'art' is a verb, representing the act of creating, rather than the static pieces found in galleries or museums.
• 🌊 The project 'Riverless Walk' was an extensive installation that allowed exploration of the Chicago Riverwalk through interactive and traditional media.
• 🖼️ The use of black and white photography in 'Riverless Walk' served as an accessible entry point for viewers to understand the subject matter before moving to more abstract elements.
• ✂️ 3D printed sculptures were created using a depth extruder algorithm that translated pixel brightness into 3D space, drawing from landscape painting techniques.
• 🏙️ The interactive installation space in 'Riverless Walk' used a rotating projector to map physical movements to a virtual space, creating a dynamic experience.
• 💻 The speaker's experience with interactive art and natural user interfaces led to a job opportunity at a startup, showcasing the practical applications of their artistic and technical skills.
• 🌱 The speaker encourages finding connections between seemingly unrelated aspects of one's life, as these intersections can lead to innovation and personal growth.

Q & A

What was the speaker's initial struggle before combining sculpture and computer science?

The speaker initially struggled to make real strides in sculpture and computer science until they started mixing the two disciplines.

What did the speaker learn from creating their first interactive piece called 'Allies'?

Creating 'Allies' taught the speaker more about creating a minimum viable product and software than any computer science class they had taken.

What does the speaker mean when they say 'art is a verb'?

The speaker means that art is the act of creating, not the finished piece displayed in a gallery or museum.

How does the speaker relate the two-week cycles in software development to art?

The speaker relates the two-week cycles in software development, known as agile development, to the bi-weekly critiques in art, suggesting both fields benefit from regular cycles of goal setting and evaluation.

What was the concept behind the speaker's installation 'Riverless Walk'?

The concept of 'Riverless Walk' was to allow people to explore the architecture of the Chicago Riverwalk through three different mediums: an interactive installation, traditional media, and black and white photography.

How does the speaker use technology in their artwork?

The speaker uses technology by creating 3D sculptures from photographs using a depth extruder algorithm and integrating interactive elements that respond to physical movement.

What is the significance of the depth extruder algorithm in the speaker's work?

The depth extruder algorithm is significant as it transforms 2D photographs into 3D space based on pixel brightness, allowing the creation of sculptures and interactive installation elements.

How does the speaker's interactive installation 'Riverless Walk' translate physical actions into virtual interactions?

In 'Riverless Walk', physical actions like walking and moving are mapped to virtual interactions, affecting the viewer's navigation through an abstract representation of a physical space.

What was the speaker's goal for the interactive portion of 'Riverless Walk'?

The speaker's goal was to move viewers past instant gratification to a deeper engagement where they understand how their physical presence affects the artwork and can then focus on the content and interactions.

How did the speaker's experience with 'Riverless Walk' lead to their first software job?

The speaker's experience with designing natural user interfaces in 'Riverless Walk' led to a job offer from a startup that needed a developer familiar with cutting-edge technology.

What was the concept of the web app developed by the company Kenobi?

The web app developed by Kenobi allowed users to get real-time feedback for learning kinesthetic activities by overlaying their skeletal data with an instructor's, to correct posture and prevent bad muscle memory.

Outlines

00:00

🎨💻 Blending Art and Technology

The speaker, who has studied both sculpture and computer science, shares their journey in merging these two fields. They discuss their first interactive art piece, 'Allies,' which was pivotal in their understanding of creating a minimum viable product and software. The speaker emphasizes the importance of the two-week development cycle, known as agile development in software, and how it parallels the critique cycles in art. They introduce the concept of 'Riverless Walk,' an interactive installation that explores the architecture of the Chicago Riverwalk through various media, including an interactive installation, traditional media, and black and white photography. The speaker also explains how they used a depth extruder algorithm to transform photographs into 3D sculptures, drawing a connection between landscape painting and modern technology.

05:02

🌐 Interactive Installations and Natural User Interfaces

The speaker delves into the technical aspects of their interactive installation, 'Riverless Walk,' describing the use of a laptop, projector, and Microsoft Kinect to create a virtual space that responds to physical movement. They discuss the importance of transitional media in art and how it helps viewers navigate between the physical and virtual worlds. The speaker also talks about their favorite part of the installation, where quick movements cause the digital representation of buildings to 'burst apart,' creating a unique visual effect. They share their experience of working at a startup called Kenobi, where they developed a web app for real-time feedback on kinesthetic activities. The app allowed users to compare their movements with an instructor's, overlaying skeletal data to correct posture and prevent bad muscle memory. Despite Kenobi's eventual shutdown, the speaker's experience led to opportunities in New Orleans, where they continued to blend art and technology.

10:02

🔄 The Synergy of Art and Technology

In the final paragraph, the speaker reflects on the synergy between art and technology, highlighting how their experiences in both fields have enriched their career. They discuss the impact of their work on the local tech and art scene in New Orleans and encourage the audience to find connections between seemingly unrelated aspects of their lives. The speaker suggests that such interdisciplinary exploration can lead to unexpected insights and opportunities, using their own journey as an example of how art and technology can complement and enhance each other. They conclude with a call to action, urging the audience to explore the intersections of their passions and interests.

Keywords

💡Sculpture

Sculpture refers to the artistic practice of creating three-dimensional works by shaping or combining materials such as stone, metal, or wood. In the video, the speaker has studied sculpture for over a decade and integrates it with computer science to create interactive installations. The concept of sculpture is expanded beyond traditional materials to include digital and interactive elements, as seen in the creation of 3D printed sculptures from photographs.

💡Computer Science

Computer science is the study of computers and computing, including their theory, design, development, and application. The speaker's background in computer science is crucial for the development of software and algorithms that drive the interactive aspects of their art installations. This field provides the technical foundation for blending art with technology, allowing for innovative expressions like the depth extruder algorithm mentioned in the script.

💡New Media Art

New Media Art is a genre that encompasses art forms that are dependent on digital technology and digital processing. The speaker's venture into new media art led to the creation of interactive installations, which are a fusion of traditional art forms with modern technology. This keyword is central to the video's theme, showcasing how technology can be harnessed to create novel artistic experiences.

💡Interactive Installations

Interactive installations are art pieces that engage the viewer by allowing them to interact with the artwork, often through physical actions or digital interfaces. The speaker's 'Riverless Walk' is an example of an interactive installation that uses a combination of physical space, digital projections, and motion tracking to create an immersive experience. This concept is integral to the video's narrative, demonstrating the practical application of blending art and technology.

💡Human-Computer Interaction (HCI)

Human-Computer Interaction is a field of study that focuses on the design and use of computer technology, focusing on the interfaces between people and computers. The speaker's exploration of HCI is evident in the design of their interactive art pieces, which require the viewer to interact with the technology to experience the art. This keyword is significant as it bridges the gap between the technical and artistic aspects of the speaker's work.

💡Agile Development

Agile development is a software development methodology that emphasizes incremental progress and flexibility. The speaker compares the iterative process of creating art to agile development, where goals are set and reassessed over two-week cycles. This keyword is used to illustrate the parallels between the creative process in art and the structured approach in software development.

💡3D Printing

3D printing is a process of creating three-dimensional objects from digital models by depositing successive layers of material. In the video, the speaker uses 3D printing to create sculptures from photographs, demonstrating a novel application of this technology in the context of art. This keyword highlights the integration of advanced manufacturing techniques into the artistic process.

💡Depth Extruder Algorithm

The depth extruder algorithm is a custom software tool used by the speaker to convert 2D images into 3D models based on pixel brightness. This algorithm is a key technical component that enables the transition from traditional photography to 3D sculptures, showcasing the intersection of computer science and sculpture in the speaker's work.

💡Natural User Interface (NUI)

A Natural User Interface is an interface that allows for intuitive interactions with technology, as opposed to traditional interfaces that require learning specific commands or gestures. The speaker's work in designing NUIs for their art installations led to a job opportunity in software development. This keyword is significant as it represents the practical application of the speaker's artistic explorations in the tech industry.

💡Chicago Riverwalk

The Chicago Riverwalk is a public pedestrian waterfront along the Chicago River, which serves as the inspiration and subject for the speaker's 'Riverless Walk' installation. This keyword is central to the video's narrative, as it is the real-world location that the speaker's interactive art aims to represent and explore through various media.

💡Kinesthetic Learning

Kinesthetic learning is a learning style that involves physical movement and tactile experiences. The speaker's work at Kenobi involved developing a web app for real-time feedback in kinesthetic activities, such as yoga or dance. This keyword is relevant as it connects the speaker's artistic explorations with their professional work in software development, emphasizing the practical applications of technology in learning and training.

Highlights

The speaker has been studying sculpture and computer science for over a decade.

They started making strides by combining their skills in both fields.

Professors encouraged the integration of computer science into art.

The speaker's first interactive piece, 'Allies', taught them about creating a minimum viable product.

They discovered human-computer interaction and interface design through their art.

Art and software both work best on two-week cycles, known as agile development in software.

The speaker's sculpture professor pointed out the similarity between art critiques and agile development.

The piece 'Riverless Walk' allowed exploration of the Chicago Riverwalk through different mediums.

The installation used black and white photography to introduce viewers to the subject matter.

Small-scale sculptures made with a 3D printer represented a semi-traditional medium.

A depth extruder algorithm was used to convert 2D photographs into 3D space.

The interactive installation space used a rotating projector to map physical actions to virtual interactions.

The speaker embraced the analog artifact of the moving projector to enhance the piece.

The goal was to move beyond instant gratification to a deeper understanding of the interactions.

The speaker's work led to a job designing and developing natural user interfaces.

They worked at a startup called Kenobi, developing a web app for real-time feedback on kinesthetic activities.

The app overlaid the user's skeletal data with an instructor's to provide feedback.

The speaker encourages finding connections between seemingly unrelated aspects of life.

They believe such connections can lead to surprising and valuable results.

Transcripts

foreign

i've been studying both sculpture and

computer science for over a decade now

but i didn't start making real strides

and either until i started mixing them

together

thanks to a series of professors

encouraging me to use my skills in

computer science or art in the other

discipline

i got into making new media art and

interactive installations

my first interactive piece

allies

taught me more about creating a minimum

viable product and software

than any computer science class i had

taken

for the first time i felt like i was

onto something

something new and worth examining

i had stumbled onto my first taste

of human computer interaction and

designing interfaces

during a bi-weekly critique in my senior

project sculpture class here at lsu i

had a eureka moment connecting software

and art time out real quick hang on

when i say art

i do not mean a piece of art to me art

is a verb it is the act of creating

the thing that you see in a gallery or a

museum

that's not art that is a piece of art or

a work of art

okay

side note noted time in

i was so excited by my profound at least

to me revelation connecting software and

art i had to share with everyone in my

class right then

so i explained to everyone

that both software and art work best on

two week cycles in the software world

this is known as agile development

you set a list a goal set a list of

goals for the next two weeks this list

of goals is called a burn down chart

you then put your head down and go to

work crossing things off your chart as

you go

at the end of the two weeks you stop and

analyze how successful you were at

completing those goals

you then set new goals based upon this

knowledge

you just keep repeating this

until you're done or more likely your

company runs out of money

so everyone just stared at me with this

thoroughly unimpressed look

my sculpture professor responds of

course cole that's why we have bye week

critiques i know

but this thing that we do as artists

has been validated by a completely

different group of people

that actually have metrics for success

no one seemed to care very much so

eventually i gave up and we all got back

to talking art

all of you should care a lot

humor me at least for the next five

minutes since that's what i'm here to

talk about today

the connections between art

and software

so the piece i was working on during

senior project was called riverless walk

it's the largest and most involved

installation i've made to date

the concept of riverless walk was to

allow people to explore the beautiful

architecture of the chicago riverwalk

through three different mediums

namely

an interactive installation that allowed

you to explore a real place that was

supported by traditional media

the space the installation was going in

had a long walkway

followed by an entrance foyer that led

into the actual room itself

this physical transitional space

provided me a beautiful vehicle for the

transitions taking place in the media of

the piece

also thanks to the confidence provided

to me by the unique technology aspect i

was able to show a media that i don't

traditionally work in

black and white photography

by starting with black and white

photography the viewer is able to

understand

immediately the subject matter of the

installation

now they're primed and ready to make

more abstract connections

moving from photography

to small-scale sculpture we move both

further away from traditional art forms

and direct representation

these small-scale sculptures are a

semi-traditional media and that

small-scale sculpture has been around

for thousands of years

but

these have a uniqueness to them and that

they were created via a 3d printer

not a block of marble and a chisel

this is also where the technology starts

the way that i made these sculptures was

by feeding the photographs into a depth

extruder algorithm

that extruded the photographs into 3d

space

based on pixel brightness

this algorithm borrows from the

tradition of landscape painting

landscape painters paint objects in the

foreground

brighter and with more color and the

closer you get to the horizon the darker

or grayer they become

the evolution of two-point perspective

came about in the renaissance thanks to

artists who also studied mathematics

namely geometry

they observed principles in one subject

and applied them in another

the most advanced technology of their

day

was mathematics the most advanced

technology of my day was the computer

so after feeding my images into this

depth extruder algorithm i got out these

3d contour lines

i was then able to stitch these lines

together to make solid printable

sculptures

i then also

reused this same algorithm in the

interactive portion of the installation

to make up the blocks that make up the

buildings so now the viewer has a point

of departure from the physical world to

the virtual world

having moved from

traditional to

semi-traditional we're now ready for the

part where art really meets tech

walking into the active

installation space

you enter a three-quarter circular room

there's a large white concrete pole

right in the middle there's a radius of

about 20 feet

i mounted directly in front of this

white concrete pole

a laptop

projector and microsoft connect all in

this custom design platform

hung around you is an 80 foot long

canvas and a half circle

and as you walk around the projector

rotates to follow you

by

moving through a physical space

you're navigating a virtual space

that is an abstract representation of a

physical space do you now see why i felt

the need for transitional media

this whole piece dealt with transitions

in space

and imagery

walking down the chicago river walk some

of these buildings are over a quarter

mile apart

i needed a way to express these

transitions to the viewer

in the interactive portion of the

installation

every

physical action maps to a virtual

interaction

so as you can see

walk right the buildings go right

get further away

the buildings get further away

come back around

they come back around and then if you

move around really quickly

everything kind of bursts apart this was

my favorite part

i noticed whenever i was developing the

installation this

analog artifact from the way the moving

projector across the canvas made the

whole thing kind of shimmer

think lens flare here

so rather than try and deny this digital

the analog artifact i embraced it it

made it part of the piece so now the

quicker you move around the more the

sides of the blocks that make up the

building come apart from one another

i wanted people to

spend time with this piece to form a

relationship

and when they understand how their

physical presence

is affecting it then they can focus on

the content and the interactions taking

place

my goal with the interactive

portion was that they would move past

the instant gratification of oh i do

this it does that to i am doing this so

i can examine that

my

previous interactive piece

to riverless walk

building bridges out of buildings led to

my first software job

designing and developing natural user

interfaces

natural user interface is a term that

relates to designing interfaces that are

intuitive and natural rather than

predefined or metaphorical

the mouse and touch screen those are

examples of a graphical user interface

that have a predefined way of

interacting with them that the user must

learn

so

thanks to countless hours of coding 100

plus on this one piece and experience

with designing natural user interfaces i

got contacted by a startup out of new

orleans

that needed not just any developer but

someone that could write code for the

microsoft connect

and

apparently i was the only person in

south louisiana doing this at the time

artist right

so since i wasn't concerned with

getting my system admin chops up to

snuff or reviewing best practices for

back end application development i

landed an awesome job writing really

interesting code

so

what we're talking tech let's take a

closer look at this the company i

started working for was called kenobi

we were developing a web app that

allowed you to get real-time feedback

for learning any kind of kinesthetic

activity think yoga golf dance

whatever uses your body we got it

covered

our app

put your video beside the instructor's

video

but then furthermore it overlaid the

instructor's skeletal data on top of

your own skeletal data so that you could

see how off you were

relative to the instructor

this is huge

because if you're trying to teach

yourself something like yoga

if you're practicing the pose wrong

that can be worse than not practicing at

all it creates bad muscle memory which

is nearly impossible to overcome

so if you're observing this carefully

you've noticed i've used the past tense

throughout here

well yes kenobi went the way of many

startups and stopped

although it stopped it got me to new

orleans which has served as a wonderful

test kitchen for evolving both my art

and software together and separately

thanks to art feeding tech and tech

feeding art i've been approached by

people starting software companies that

need not just any developer

but someone who's familiar with cutting

edge and often buggy tech

new orleans has always been known for

food music and art there's currently a

large group of us that are working on

adding technology to that mixture and i

couldn't be happier to be contributing

my own special blend to the pot

but

i would like to finish my talk with a

request to y'all

find a way to connect the seemingly

unrelated aspects of your life

you will be amazed at the results

maybe you're a chef by day and a gear

head by night

i bet there's something to be learned by

taking your skillet into the garage one

night

i don't know what you'll learn

to be fair i don't know if you'll learn

anything at all

maybe some things connect better than

others and i just got really lucky with

software and art

but

i would be willing to wager it's not a

totally unique situation

look for the little things at first like

the two-week cycles i noticed connecting

art and software

once you can find one really solid

branch

everything else will start connecting

thanks