Linear combinations, span, and basis vectors | Chapter 2, Essence of linear algebra

Summary

This video delves into the concept of vector coordinates, representing vectors as linear combinations of basis vectors. It explores how any 2D vector can be expressed as a sum of scaled versions of the standard basis vectors (i and j). The notion of span is introduced, which represents all possible linear combinations of a set of vectors. Linear independence and dependence are discussed, leading to the definition of a basis as a set of linearly independent vectors that span the space. The video provides insightful visualizations and thought-provoking puzzles, fostering a deeper understanding of these fundamental linear algebra concepts.

Takeaways

• 😀 Vector coordinates represent vectors as pairs of numbers, with each coordinate acting as a scalar that stretches or squishes basis vectors (i-hat and j-hat in 2D).
• 🤔 Any pair of 2D vectors (except when they are co-linear) can span the entire 2D plane through linear combinations, while co-linear vectors span only a line.
• ➕ Linear combinations involve scaling vectors by scalars and adding them together, enabling the construction of new vectors from a set of basis vectors.
• 🌐 The span of a set of vectors is the collection of all possible vectors that can be represented as linear combinations of those vectors.
• 🧭 Linearly independent vectors are those that each contribute a new dimension to the span, while linearly dependent vectors are redundant and can be expressed as linear combinations of the others.
• ✨ A basis is a set of linearly independent vectors that can span the entire space through linear combinations.
• 🔢 Different choices of basis vectors lead to different coordinate systems for representing vectors numerically.
• 📐 In 3D, the span of two non-collinear vectors is a flat sheet, while the span of three general vectors can reach the entire 3D space.
• 🌉 The concept of linear combinations and spans provides a way to understand and visualize the relationships between vectors in a space.
• 🧩 Understanding linear independence, span, and basis is fundamental to the study of linear algebra and its applications.

Q & A

What is the purpose of discussing vector coordinates in the video?

The video discusses vector coordinates to show the connection between pairs of numbers and two-dimensional vectors, and to introduce the concept of basis vectors and linear combinations.

What are the basis vectors i-hat and j-hat, and how are they related to vector coordinates?

i-hat and j-hat are unit vectors pointing in the x and y directions, respectively. The coordinates of a vector are scalars that scale these basis vectors, and the vector itself is the sum of the scaled basis vectors.

Can we choose different basis vectors other than i-hat and j-hat?

Yes, we can choose any two non-collinear vectors as the basis vectors for a two-dimensional coordinate system. This will result in a different association between vector coordinates and actual vectors.

What is a linear combination of vectors?

A linear combination of vectors is obtained by scaling each vector by a scalar and then adding the scaled vectors together.

What is the span of a set of vectors?

The span of a set of vectors is the set of all possible vectors that can be obtained as linear combinations of those vectors.

What happens to the span of two vectors in 3D space?

The span of two non-collinear vectors in 3D space is a flat sheet or plane passing through the origin.

How does adding a third vector affect the span in 3D space?

If the third vector is not in the plane spanned by the first two vectors, then the span becomes the entire 3D space. If the third vector lies in the same plane, the span remains the same.

What does it mean for a set of vectors to be linearly dependent or linearly independent?

A set of vectors is linearly dependent if one of the vectors can be expressed as a linear combination of the others. If no vector can be expressed as a linear combination of the others, they are linearly independent.

What is the technical definition of a basis for a space?

A basis of a space is a set of linearly independent vectors that span that space.

Why does the technical definition of a basis make sense, according to the video?

The video does not explicitly explain why this definition makes sense, but it is likely because a basis should consist of vectors that are not redundant (linearly independent) and can collectively access the entire space (span the space).

Outlines

00:00

🧮 Vector Coordinates and Basis Concepts

This section delves into the concept of vector coordinates and their significance in linear algebra, emphasizing the interpretation of coordinates as scalars that stretch or squish vectors. It introduces the basis vectors i-hat and j-hat in the xy coordinate system, explaining how vector coordinates scale these basis vectors to represent different vectors through their sum. The notion of basis vectors as fundamental to understanding coordinates is discussed, highlighting their role in scaling operations. The idea of varying basis vectors to create different, yet valid, coordinate systems is also explored, prompting a reflection on the versatility and implications of this concept. The discussion extends to linear combinations of vectors and their association with linear spaces, introducing terms like 'span' and 'linearly independent/dependent' to describe the reach and relationships of vector sets. The narrative encourages contemplating the implications of choosing different basis vectors and the mathematical structures they unveil.

05:00

📐 Span, Linear Independence, and Three-Dimensional Vectors

This paragraph explores the concept of span in both two-dimensional and three-dimensional spaces, using vivid imagery to describe how vectors combine linearly to fill spaces. It contrasts the infinite possibilities offered by non-aligned vectors with the limitations of linearly dependent vectors, which confine the span to a flat sheet or a line. The addition of a third vector in three-dimensional space is examined for its potential to expand the span to encompass all possible vectors in that space, provided it is not linearly dependent on the first two. The terminology of linear dependence and independence is explained in the context of vector spans and their dimensions. The discussion prepares the ground for understanding the definition of a basis as a set of linearly independent vectors that span a given space, tying back to the significance of basis vectors introduced in the first paragraph. It concludes by setting up a puzzle that connects the discussed concepts of basis, span, and linear independence, laying the groundwork for future exploration of matrices and space transformation.

Keywords

💡Vector Coordinates

Vector coordinates refer to representing vectors using pairs (or triplets for 3D) of numbers, allowing a back-and-forth translation between numerical values and geometric vectors. The video highlights the importance of this concept in linear algebra, relating the coordinates to scaling basis vectors like i-hat and j-hat.

💡Basis Vectors

Basis vectors are a set of special vectors (like i-hat and j-hat in 2D, or i-hat, j-hat, k-hat in 3D) that form the foundational coordinate system. Vector coordinates describe a vector by scaling and summing these basis vectors. The choice of basis vectors determines the specific coordinate system used.

💡Linear Combination

A linear combination is a vector obtained by scaling multiple vectors and adding them together. Specifically, it involves multiplying each vector by a scalar (numerical value) and summing the results. Linear combinations allow expressing any vector in terms of a set of basis vectors using their coordinates.

💡Span

The span of a set of vectors is the collection of all possible linear combinations of those vectors. It represents the full range or space of vectors that can be reached by scaling and adding the given set of vectors. The concept of span relates to what portion of the coordinate space is accessible using a particular set of basis vectors.

💡Linearly Dependent

A set of vectors is linearly dependent if one or more vectors in the set can be expressed as a linear combination of the others. In other words, some vectors are redundant and do not contribute new directions to the span of the set. The video uses this term to describe situations where adding an extra vector does not increase the accessible space.

💡Linearly Independent

A set of vectors is linearly independent if no vector in the set can be expressed as a linear combination of the others. Each vector contributes a new direction, expanding the span of the set. Linearly independent vectors are essential for forming a basis that can access the full coordinate space.

💡Basis

A basis is a set of linearly independent vectors that span the entire space or coordinate system. It is a minimal set of vectors that can express any vector in that space through linear combinations using their coordinates. The standard bases (like i-hat, j-hat in 2D) are commonly used, but other choices are possible.

💡Scalar Multiplication

Scalar multiplication is the operation of multiplying a vector by a scalar (numerical) value, resulting in a vector with the same direction but a scaled magnitude. This concept is fundamental in linear algebra, as it allows scaling basis vectors to construct linear combinations representing any vector in the coordinate space.

💡Vector Addition

Vector addition is the operation of combining two vectors by adding their corresponding components (coordinates) together. It is a crucial concept in linear algebra, as it allows constructing new vectors from existing ones through linear combinations, thereby accessing different points in the coordinate space.

💡Linear

The term 'linear' in linear algebra refers to the fact that the operations involved (vector addition and scalar multiplication) preserve the property of lying on a straight line. When one scalar is fixed in a linear combination, the resulting vectors trace out a line. The 'linear' aspect captures this relationship with lines and planes.

Highlights

Vector coordinates represent vectors using pairs of numbers, with each coordinate acting as a scalar that stretches or squishes basis vectors (i-hat and j-hat in the xy coordinate system).

Different choices of basis vectors lead to different coordinate systems for representing vectors.

A linear combination of vectors involves scaling each vector by a scalar and adding the scaled vectors together.

The span of a set of vectors is the set of all possible linear combinations of those vectors.

For most pairs of 2D vectors, the span is the entire 2D plane, but if the vectors are linearly dependent (aligned), the span is a line.

The span of two non-parallel 3D vectors is a flat sheet passing through the origin.

Adding a third 3D vector not in the span of the first two vectors allows access to the entire 3D space.

Vectors are linearly dependent if one can be expressed as a linear combination of the others, making it redundant in terms of spanning the space.

Vectors are linearly independent if each one adds a new dimension to the span.

A basis is a set of linearly independent vectors that span the entire space.

The mental image of spanning vectors by adjusting scalars (knobs) and tracing the tip of the resulting vector is described as beautiful.

The etymology of the term "linear" in linear combinations is related to the resulting vector drawing a straight line when one scalar is fixed and the other varies.

Representing vectors as points rather than arrows can be convenient when dealing with collections of vectors.

The lecture builds an intuitive understanding of linear algebra concepts like linear combinations, span, linear dependence/independence, and basis through visual examples and analogies.

The lecture leaves a puzzle about why the technical definition of a basis (a set of linearly independent vectors that span the space) makes sense, setting up the next topic.

Transcripts

In the last video, along with the ideas of vector addition and scalar multiplication,

I described vector coordinates, where there's this back and forth between,

for example, pairs of numbers and two-dimensional vectors.

Now, I imagine the vector coordinates were already familiar to a lot of you,

but there's another kind of interesting way to think about these coordinates,

which is pretty central to linear algebra.

When you have a pair of numbers that's meant to describe a vector,

like 3, negative 2, I want you to think about each coordinate as a scalar,

meaning, think about how each one stretches or squishes vectors.

In the xy coordinate system, there are two very special vectors,

the one pointing to the right with length 1, commonly called i-hat,

or the unit vector in the x direction, and the one pointing straight up with length 1,

commonly called j-hat, or the unit vector in the y direction.

Now, think of the x coordinate of our vector as a scalar that scales i-hat,

stretching it by a factor of 3, and the y coordinate as a scalar that scales j-hat,

flipping it and stretching it by a factor of 2.

In this sense, the vector that these coordinates

describe is the sum of two scaled vectors.

That's a surprisingly important concept, this idea of adding together two scaled vectors.

Those two vectors, i-hat and j-hat, have a special name, by the way.

Together, they're called the basis of a coordinate system.

What this means, basically, is that when you think about coordinates as scalars,

the basis vectors are what those scalars actually, you know, scale.

There's also a more technical definition, but I'll get to that later.

By framing our coordinate system in terms of these two special basis vectors,

it raises a pretty interesting and subtle point.

We could have chosen different basis vectors and

gotten a completely reasonable new coordinate system.

For example, take some vector pointing up and to the right,

along with some other vector pointing down and to the right in some way.

Take a moment to think about all the different vectors that you can get by choosing two

scalars, using each one to scale one of the vectors, then adding together what you get.

Which two-dimensional vectors can you reach by altering the choices of scalars?

The answer is that you can reach every possible two-dimensional vector,

and I think it's a good puzzle to contemplate why.

A new pair of basis vectors like this still gives us a valid way to go back and forth

between pairs of numbers and two-dimensional vectors,

but the association is definitely different from the one that you get using the more

standard basis of i-hat and j-hat.

This is something I'll go into much more detail on later,

describing the exact relationship between different coordinate systems,

but for right now, I just want you to appreciate the fact that any time we

describe vectors numerically, it depends on an implicit choice of what basis

vectors we're using.

So any time that you're scaling two vectors and adding them like this,

it's called a linear combination of those two vectors.

Where does this word linear come from?

Why does this have anything to do with lines?

Well, this isn't the etymology, but one way I like to think about it

is that if you fix one of those scalars and let the other one change its value freely,

the tip of the resulting vector draws a straight line.

Now, if you let both scalars range freely and consider every possible

vector that you can get, there are two things that can happen.

For most pairs of vectors, you'll be able to reach every possible point in the plane.

Every two-dimensional vector is within your grasp.

However, in the unlucky case where your two original vectors happen to line up,

the tip of the resulting vector is limited to just this single line passing through the

origin.

Actually, technically there's a third possibility too.

Both your vectors could be zero, in which case you'd just be stuck at the origin.

Here's some more terminology.

The set of all possible vectors that you can reach with a linear combination

of a given pair of vectors is called the span of those two vectors.

So restating what we just saw in this lingo, the span of most

pairs of 2D vectors is all vectors of 2D space, but when they line up,

their span is all vectors whose tip sit on a certain line.

Remember how I said that linear algebra revolves

around vector addition and scalar multiplication?

Well, the span of two vectors is basically a way of asking what

are all the possible vectors you can reach using only these two fundamental operations,

vector addition and scalar multiplication.

This is a good time to talk about how people commonly think about vectors as points.

It gets really crowded to think about a whole collection of vectors sitting on a line,

and more crowded still to think about all two-dimensional vectors all at once,

filling up the plane.

So when dealing with collections of vectors like this,

it's common to represent each one with just a point in space,

the point at the tip of that vector where, as usual,

I want you thinking about that vector with its tail on the origin.

That way, if you want to think about every possible vector whose

tip sits on a certain line, just think about the line itself.

Likewise, to think about all possible two-dimensional vectors all at once,

conceptualize each one as the point where its tip sits.

So in effect, what you'll be thinking about is the infinite flat

sheet of two-dimensional space itself, leaving the arrows out of it.

In general, if you're thinking about a vector on its own, think of it as an arrow.

And if you're dealing with a collection of vectors,

it's convenient to think of them all as points.

So for our span example, the span of most pairs of vectors ends

up being the entire infinite sheet of two-dimensional space.

But if they line up, their span is just a line.

The idea of span gets a lot more interesting if we

start thinking about vectors in three-dimensional space.

For example, if you take two vectors in 3D space that are not

pointing in the same direction, what does it mean to take their span?

Well, their span is the collection of all possible linear combinations

of those two vectors, meaning all possible vectors you get by scaling

each of the two of them in some way and then adding them together.

You can kind of imagine turning two different knobs to change

the two scalars defining the linear combination,

adding the scaled vectors and following the tip of the resulting vector.

That tip will trace out some kind of flat sheet

cutting through the origin of three-dimensional space.

This flat sheet is the span of the two vectors.

Or more precisely, the set of all possible vectors whose

tips sit on that flat sheet is the span of your two vectors.

Isn't that a beautiful mental image?

So, what happens if we add a third vector and

consider the span of all three of those guys?

A linear combination of three vectors is defined

pretty much the same way as it is for two.

You'll choose three different scalars, scale each of those vectors,

and then add them all together.

And again, the span of these vectors is the set of all possible linear combinations.

Two different things could happen here.

If your third vector happens to be sitting on the span of the first two,

then the span doesn't change.

You're sort of trapped on that same flat sheet.

In other words, adding a scaled version of that third vector to the

linear combination doesn't really give you access to any new vectors.

But if you just randomly choose a third vector,

it's almost certainly not sitting on the span of those first two.

Then, since it's pointing in a separate direction,

it unlocks access to every possible three-dimensional vector.

One way I like to think about this is that as you scale that new third vector,

it moves around that span sheet of the first two, sweeping it through all of space.

Another way to think about it is that you're making full use of the three freely changing

scalars that you have at your disposal to access the full three dimensions of space.

Now, in the case where the third vector was already sitting on the span of the first two,

or the case where two vectors happen to line up,

we want some terminology to describe the fact that at least one of these vectors is

redundant, not adding anything to our span.

Whenever this happens, where you have multiple vectors and you could remove one without

reducing the span, the relevant terminology is to say that they are linearly dependent.

Another way of phrasing that would be to say that one of the vectors can be expressed

as a linear combination of the others, since it's already in the span of the others.

On the other hand, if each vector really does add another dimension to the span,

they're said to be linearly independent.

So with all of that terminology, and hopefully with some good mental

images to go with it, let me leave you with a puzzle before we go.

The technical definition of a basis of a space is a

set of linearly independent vectors that span that space.

Now, given how I described a basis earlier, and given your current understanding of the

words span and linearly independent, think about why this definition would make sense.

In the next video, I'll get into matrices in transforming space.