Abstract vector spaces | Chapter 16, Essence of linear algebra

Summary

This video script delves into the fundamental nature of vectors, exploring whether they are best understood as arrows in a plane or as lists of numbers. It challenges the viewer to consider vectors as spatial entities or manifestations of a deeper concept. The script introduces an analogy between vectors and functions, demonstrating how functions can be treated as vectors by applying principles of linear algebra. Through the lens of linear transformations and the properties of additivity and scaling, the video builds a bridge between abstract mathematical concepts and concrete applications. The script concludes by emphasizing the abstract nature of vectors in modern linear algebra, defined by a set of axioms that allow for a wide range of 'vectorish' objects, including arrows, numbers, functions, and more.

Takeaways

• 🔍 Vectors can be viewed in multiple ways, such as arrows in a plane or pairs of real numbers, and may have deeper spatial properties.
• 📏 Defining vectors as lists of numbers is clear-cut, but they may represent an independent space that coordinates are arbitrarily assigned to.
• 📊 Core linear algebra topics like determinants and eigenvectors are spatial and invariant under changes of coordinate systems.
• 🎯 The essence of vectors might be more about the operations of addition and scaling rather than their representation as numbers or arrows.
• 📚 Functions can also be considered vectors because they support addition and scaling, similar to vectors in linear algebra.
• ✏️ Linear transformations, like the derivative in calculus, can be applied to functions, emphasizing the vector-like qualities of functions.
• 📐 A linear transformation must satisfy additivity and scaling to be considered linear, preserving the operations of vector addition and scalar multiplication.
• 📈 The concept of a matrix can be extended to function spaces, such as polynomials, to describe operations like derivatives.
• 📘 Vector spaces are sets of objects that adhere to the rules of vector addition and scalar multiplication, forming the abstract foundation of linear algebra.
• 📝 Axioms in linear algebra define the properties that vector addition and scalar multiplication must follow, allowing for general application across different types of vector spaces.
• 🎓 The modern theory of linear algebra focuses on abstraction, allowing mathematicians to apply their results to any vector space that satisfies the defined axioms.

Q & A

What is the deceptively simple question the video aims to revisit?

The video revisits the question of what vectors are, exploring whether they are fundamentally arrows on a flat plane, pairs of real numbers, or manifestations of something deeper.

Why might defining vectors as primarily a list of numbers feel clear-cut and unambiguous?

Defining vectors as a list of numbers provides a clear and unambiguous way to conceptualize higher-dimensional vectors, making abstract ideas like four-dimensional vectors more concrete and workable.

What is the common sensation among those who work with linear algebra as they become more fluent with changing their basis?

The common sensation is that they are dealing with a space that exists independently from the coordinates, and that coordinates are somewhat arbitrary, depending on the chosen basis vectors.

Why do determinants and eigenvectors seem indifferent to the choice of coordinate systems?

Determinants and eigenvectors are core topics in linear algebra that are inherently spatial, and their underlying values do not change with different coordinate systems.

How does the concept of functions relate to the concept of vectors?

Functions can be seen as another type of vector because they can be added together and scaled by real numbers, similar to how vectors can be combined through addition and scalar multiplication.

What is the formal definition of a linear transformation for functions?

A linear transformation for functions is defined by two properties: additivity and scaling. Additivity means the transformation of the sum of two functions is the same as the sum of their individual transformations. Scaling means scaling a function and then transforming it yields the same result as transforming the function first and then scaling the result.

Why is the concept of a basis important when dealing with function spaces?

A basis is important because it provides a coordinate system for the function space, allowing for the representation of functions as vectors with coordinates, which is essential for applying linear algebra concepts to functions.

How is the derivative of a function related to the concept of linear transformations?

The derivative is an example of a linear transformation for functions. It takes one function and turns it into another while preserving the properties of additivity and scaling.

What is the significance of the matrix representation of the derivative in the context of polynomials?

The matrix representation of the derivative for polynomials allows for the application of linear algebra techniques to function spaces, specifically polynomial functions, making it possible to visualize and calculate derivatives using matrix-vector multiplication.

What are vector spaces and why are they fundamental in the theory of linear algebra?

Vector spaces are sets of objects that follow certain rules for vector addition and scalar multiplication, known as axioms. They are fundamental because they provide a general framework for applying linear algebra concepts to various 'vectorish' things, regardless of their specific form or nature.

How does the modern theory of linear algebra approach the concept of vectors?

The modern theory of linear algebra approaches vectors abstractly, focusing on the axioms that define vector spaces rather than on the specific form vectors take, such as arrows, lists of numbers, or functions.

Why is it beneficial to start learning linear algebra with a concrete, visualizable setting?

Starting with a concrete, visualizable setting, like 2D space with arrows, helps build intuitions about linear algebra concepts. These intuitions can then be applied more efficiently to more abstract or complex scenarios as one progresses in their understanding of the subject.

Outlines

00:00

🔍 Exploring the Essence of Vectors

This paragraph delves into the fundamental nature of vectors, questioning whether they are best understood as arrows in a plane with coordinates or as pairs of real numbers. It explores the idea that vectors might be manifestations of a deeper spatial concept, independent of the coordinate system. The paragraph also introduces the notion that functions can be treated similarly to vectors, with operations like addition and scaling, and suggests that linear algebra's principles can be applied to functions, like the derivative, which is a linear transformation.

05:05

📚 Linearity in Functions and Vectors

The second paragraph discusses the concept of linearity, specifically in the context of transformations applied to vectors and functions. It explains that a transformation is linear if it maintains the properties of additivity and scaling. The paragraph uses the derivative as an example of a linear transformation for functions and illustrates how this can be represented using an infinite matrix when dealing with polynomials. It emphasizes the parallel between the operations on vectors and functions, highlighting the universality of linear algebra concepts.

10:07

🌐 The Abstract Concept of Vector Spaces

This paragraph introduces the abstract concept of vector spaces, which are sets of objects that can be scaled and added together while adhering to certain axioms. It explains that mathematicians use these axioms to ensure that their discoveries in linear algebra are applicable to any vector space that follows the rules, regardless of the specific nature of the vectors involved. The paragraph also discusses the importance of abstract reasoning in mathematics, allowing for broad application of concepts without being confined to a specific representation of vectors.

15:09

🎓 Embracing Abstraction in Linear Algebra

The final paragraph concludes the series on the essence of linear algebra by emphasizing the importance of understanding the abstract nature of vectors and vector spaces. It suggests that while concrete, visualizable examples are helpful for beginners, the true power of linear algebra comes from its broad applicability to any vector space that follows the established axioms. The paragraph encourages learners to apply their intuitions about vectors to more complex problems and wishes them well in their future studies.

Keywords

💡Vectors

Vectors are mathematical objects that can be described as arrows with direction and magnitude, or as pairs of real numbers. In the context of the video, vectors are explored as having a deeper spatial essence rather than just being lists of numbers. The script discusses vectors in relation to linear algebra, emphasizing their role in transformations and coordinate systems. For example, the video mentions two-dimensional vectors as arrows on a flat plane and four-dimensional vectors as abstract concepts.

💡Linear Algebra

Linear algebra is a branch of mathematics that deals with the study of vectors, vector spaces (also called linear spaces), and linear transformations. The video script delves into the core topics of linear algebra, such as determinants and eigenvectors, and how they are indifferent to the choice of coordinate systems. Linear algebra provides a framework for understanding and working with vectors beyond their geometric representation.

💡Coordinates

Coordinates are numerical values that define the position of a point in a space. In the video, coordinates are discussed as a means of describing vectors for convenience, but also as somewhat arbitrary depending on the chosen basis vectors. The script suggests that the space a vector represents exists independently of the coordinates assigned to it.

💡Basis Vectors

Basis vectors are a set of vectors that define the direction and scale of a vector space. They are fundamental in establishing a coordinate system. The video script explains that changing the basis vectors can alter the coordinates of a vector, but the underlying spatial essence of the vector remains the same.

💡Determinants

In linear algebra, a determinant is a scalar value that can be computed from the elements of a square matrix, which provides information about the matrix's properties, such as its invertibility and how it scales areas. The video script mentions determinants as a core topic in linear algebra that is indifferent to the choice of coordinate systems.

💡Eigenvectors

Eigenvectors are vectors that, when a linear transformation is applied to them, remain on their own span, only being scaled. They are associated with eigenvalues, which are the scaling factors. The video script discusses eigenvectors as spatial properties that remain unchanged regardless of the coordinate system.

💡Functions

Functions are mathematical mappings from one set to another, often represented as f(x) = y. The video script introduces functions as another type of vector, where they can be added together and scaled by real numbers, similar to vectors. This concept is used to illustrate that linear algebra concepts can be applied to functions as well.

💡Linear Transformation

A linear transformation, also referred to as a linear operator in the context of functions, is a function that maps one vector space to another while preserving the operations of vector addition and scalar multiplication. The video script explains that the derivative is an example of a linear transformation for functions.

💡Additivity

Additivity is a property of a function, such as a linear transformation, where the function of the sum of two vectors is equal to the sum of the function of each vector. In the video, additivity is discussed as one of the properties that define a linear transformation, with the derivative being an example that possesses this property.

💡Scaling

Scaling refers to the operation of multiplying a vector by a scalar, which is a real number. The video script discusses the scaling property as a characteristic of linear transformations, where scaling a vector and then applying a transformation yields the same result as applying the transformation first and then scaling.

💡Vector Spaces

Vector spaces, also known as linear spaces, are mathematical structures that consist of a set of vectors and two operations, vector addition and scalar multiplication, that follow certain rules (axioms). The video script emphasizes that the concept of a vector is generalized in the modern theory of linear algebra to any set of objects that adhere to these axioms, making the form that vectors take irrelevant.

💡Axioms

Axioms are the basic principles or rules from which other truths are derived. In the context of vector spaces, the video script mentions that there are eight axioms that any vector space must satisfy for the constructs of linear algebra to apply. These axioms serve as an interface between mathematicians and others who want to apply linear algebra to new types of vector spaces.

Highlights

Vectors can be viewed as arrows in a flat plane or pairs of real numbers, or manifestations of something deeper.

Defining vectors as lists of numbers makes higher-dimensional vectors more concrete.

Linear algebra suggests vectors are spaces independent of coordinates, with coordinates being somewhat arbitrary.

Core linear algebra topics like determinants and eigenvectors are indifferent to the choice of coordinate systems.

Functions can be considered another type of vector, with operations similar to vector addition and scaling.

Linear transformations can be applied to functions, such as the derivative in calculus.

Linearity in transformations is defined by additivity and scaling properties.

A linear transformation is fully described by its effect on basis vectors.

The derivative is an example of a linear transformation that is additive and has scaling properties.

Matrix representation of the derivative can be constructed for polynomial functions.

Polynomial functions can be represented in a vector space with an infinite basis of powers of x.

An infinite matrix can describe the derivative operation in the context of polynomials.

The concept of vector spaces allows for the abstraction of different types of vectors, such as arrows, lists, or functions.

Vector spaces are sets of objects that adhere to rules of vector addition and scalar multiplication.

Axioms in linear algebra define the properties that any vector space must satisfy.

Mathematical abstraction allows for general application of linear algebra concepts across different types of vector spaces.

The modern theory of vectors focuses on the properties and operations rather than their concrete form.

Linear algebra concepts can be applied to various vectorish things following the established axioms.

The series concludes by emphasizing the importance of intuitions in understanding linear algebra.

Transcripts

I'd like to revisit a deceptively simple question

that I asked in the very first video of this series.

What are vectors?

Is a two-dimensional vector, for example, fundamentally an arrow on

a flat plane that we can describe with coordinates for convenience?

Or is it fundamentally that pair of real numbers which

is just nicely visualized as an arrow on a flat plane?

Or are both of these just manifestations of something deeper?

On the one hand, defining vectors as primarily being

a list of numbers feels clear-cut and unambiguous.

It makes things like four-dimensional vectors or 100-dimensional vectors sound like real,

concrete ideas that you can actually work with.

When otherwise, an idea like four dimensions is just a vague geometric

notion that's difficult to describe without waving your hands a bit.

But on the other hand, a common sensation for those who actually work with

linear algebra, especially as you get more fluent with changing your basis,

is that you're dealing with a space that exists independently from the

coordinates that you give it, and that coordinates are actually somewhat arbitrary,

depending on what you happen to choose as your basis vectors.

Core topics in linear algebra, like determinants and eigenvectors,

seem indifferent to your choice of coordinate systems.

The determinant tells you how much a transformation scales areas,

and eigenvectors are the ones that stay on their own span during a transformation.

But both of these properties are inherently spatial,

and you can freely change your coordinate system without changing the underlying

values of either one.

But if vectors are not fundamentally lists of real numbers,

and if their underlying essence is something more spatial,

that just begs the question of what mathematicians mean when they use a

word like space or spatial.

To build up to where this is going, I'd actually like to spend the

bulk of this video talking about something which is neither an arrow

nor a list of numbers, but also has vector-ish qualities – functions.

You see, there's a sense in which functions are actually just another type of vector.

In the same way that you can add two vectors together,

there's also a sensible notion for adding two functions, f and g,

to get a new function, f plus g.

It's one of those things where you kind of already know what it's going to be,

but actually phrasing it is a mouthful.

The output of this new function at any given input, like negative four,

is the sum of the outputs of f and g when you evaluate them each at that same input,

negative four.

Or more generally, the value of the sum function at any

given input x is the sum of the values f of x plus g of x.

This is pretty similar to adding vectors coordinate by coordinate,

it's just that there are, in a sense, infinitely many coordinates to deal with.

Similarly, there's a sensible notion for scaling a function by a real number,

just scale all of the outputs by that number.

And again, this is analogous to scaling a vector coordinate by coordinate,

it just feels like there's infinitely many coordinates.

Now, given that the only thing vectors can really do is get added together or scaled,

it feels like we should be able to take the same useful constructs and problem

solving techniques of linear algebra that were originally thought about in

the context of arrows and space and apply them to functions as well.

For example, there's a perfectly reasonable notion of a linear transformation

for functions, something that takes in one function and turns it into another.

One familiar example comes from calculus, the derivative.

It's something which transforms one function into another function.

Sometimes in this context you'll hear these called operators instead of transformations,

but the meaning is the same.

A natural question you might want to ask is what it

means for a transformation of functions to be linear.

The formal definition of linearity is relatively abstract and symbolically driven

compared to the way that I first talked about it in chapter 3 of this series.

But the reward of abstractness is that we'll get something

general enough to apply to functions as well as arrows.

A transformation is linear if it satisfies two properties,

commonly called additivity and scaling.

Additivity means that if you add two vectors, v and w,

then apply a transformation to their sum, you get the same result as if you added the

transformed versions of v and w.

The scaling property is that when you scale a vector v by some number,

then apply the transformation, you get the same ultimate vector as

if you scaled the transformed version of v by that same amount.

The way you'll often hear this described is that linear transformations

preserve the operations of vector addition and scalar multiplication.

The idea of gridlines remaining parallel and evenly spaced that I've

talked about in past videos is really just an illustration of what

these two properties mean in the specific case of points in 2D space.

One of the most important consequences of these properties,

which makes matrix vector multiplication possible,

is that a linear transformation is completely described by where it

takes the basis vectors.

Since any vector can be expressed by scaling and adding the basis vectors in some way,

finding the transformed version of a vector comes down to scaling and adding

the transformed versions of the basis vectors in that same way.

As you'll see in just a moment, this is as true for functions as it is for arrows.

For example, calculus students are always using the fact that the derivative is

additive and has the scaling property, even if they haven't heard it phrased that way.

If you add two functions, then take the derivative,

it's the same as first taking the derivative of each one separately,

then adding the result.

Similarly, if you scale a function, then take the derivative,

it's the same as first taking the derivative, then scaling the result.

To really drill in the parallel, let's see what it

might look like to describe the derivative with a matrix.

This will be a little tricky, since function spaces have a tendency to be

infinite dimensional, but I think this exercise is actually quite satisfying.

Let's limit ourselves to polynomials, things like x squared plus 3x plus 5,

or 4x to the seventh minus 5x squared.

Each of the polynomials in our space will only have finitely many terms,

but the full space is going to include polynomials with arbitrarily large degree.

The first thing we need to do is give coordinates to this space,

which requires choosing a basis.

Since polynomials are already written down as the sum of scaled powers of the variable x,

it's pretty natural to just choose pure powers of x as the basis function.

In other words, our first basis function will be the constant function, b0 of x equals 1.

The second basis function will be b1 of x equals x,

then b2 of x equals x squared, then b3 of x equals x cubed, and so on.

The role that these basis functions serve will be similar to the roles of i-hat,

j-hat, and k-hat in the world of vectors as arrows.

Since our polynomials can have arbitrarily large degree,

this set of basis functions is infinite.

But that's okay, it just means that when we treat our polynomials as vectors,

they're going to have infinitely many coordinates.

A polynomial like x squared plus 3x plus 5, for example,

would be described with the coordinates 5, 3, 1, then infinitely many zeros.

You'd read this as saying that it's 5 times the first basis function,

plus 3 times that second basis function, plus 1 times the third basis function,

and then none of the other basis functions should be added from that point on.

The polynomial 4x to the seventh minus 5x squared would have the coordinates 0,

0, negative 5, 0, 0, 0, 0, 4, then an infinite string of zeros.

In general, since every individual polynomial has only finitely many terms,

its coordinates will be some finite string of numbers with an infinite tail of zeros.

In this coordinate system, the derivative is described with

an infinite matrix that's mostly full of zeros,

but which has the positive integers counting down on this offset diagonal.

I'll talk about how you could find this matrix in just a moment,

but the best way to get a feel for it is to just watch it in action.

Take the coordinates representing the polynomial x cubed plus 5x squared plus 4x plus 5,

then put those coordinates on the right of the matrix.

The only term that contributes to the first coordinate of the result is 1 times 4,

which means the constant term in the result will be 4.

This corresponds to the fact that the derivative of 4x is the constant 4.

The only term contributing to the second coordinate of the matrix vector product

is 2 times 5, which means the coefficient in front of x in the derivative is 10.

That one corresponds to the derivative of 5x squared.

Similarly, the third coordinate in the matrix

vector product comes down to taking 3 times 1.

This one corresponds to the derivative of x cubed being 3x squared.

And after that, it'll be nothing but zeros.

What makes this possible is that the derivative is linear.

And for those of you who like to pause and ponder,

you could construct this matrix by taking the derivative of each

basis function and putting the coordinates of the results in each column.

So, surprisingly, matrix vector multiplication and taking a derivative,

which at first seem like completely different animals,

are both just really members of the same family.

In fact, most of the concepts I've talked about in this series with

respect to vectors as arrows in space, things like the dot product or eigenvectors,

have direct analogs in the world of functions,

though sometimes they go by different names, things like inner product or eigenfunction.

So back to the question of what is a vector.

The point I want to make here is that there are lots of vectorish things in math.

As long as you're dealing with a set of objects where there's a reasonable notion of

scaling and adding, whether that's a set of arrows in space, lists of numbers, functions,

or whatever other crazy thing you choose to define,

all of the tools developed in linear algebra regarding vectors,

linear transformations and all that stuff, should be able to apply.

Take a moment to imagine yourself right now as a

mathematician developing the theory of linear algebra.

You want all of the definitions and discoveries of your work to apply to

all of the vectorish things in full generality, not just to one specific case.

These sets of vectorish things, like arrows or lists of numbers or functions,

are called vector spaces.

And what you as the mathematician might want to do is say,

hey everyone, I don't want to have to think about all the

different types of crazy vector spaces that you all might come up with.

So what you do is establish a list of rules that

vector addition and scaling have to abide by.

These rules are called axioms, and in the modern theory of linear algebra,

there are eight axioms that any vector space must satisfy if all of

the theory and constructs that we've discovered are going to apply.

I'll leave them on the screen here for anyone who wants to pause and ponder,

but basically it's just a checklist to make sure that the notions of vector

addition and scalar multiplication do the things that you'd expect them to do.

These axioms are not so much fundamental rules of nature as they are an

interface between you, the mathematician, discovering results,

and other people who might want to apply those results to new sorts of vector spaces.

If, for example, someone defines some crazy type of vector space,

like the set of all pi creatures with some definition of adding and scaling pi creatures,

these axioms are like a checklist of things that they need to verify about

their definitions before they can start applying the results of linear algebra.

And you, as the mathematician, never have to think about

all the possible crazy vector spaces people might define.

You just have to prove your results in terms of these axioms so

anyone whose definitions satisfy those axioms can happily apply your results,

even if you never thought about their situation.

As a consequence, you'd tend to phrase all of your results pretty abstractly,

which is to say, only in terms of these axioms,

rather than centering on a specific type of vector, like arrows in space or functions.

For example, this is why just about every textbook you'll find will

define linear transformations in terms of additivity and scaling,

rather than talking about gridlines remaining parallel and evenly spaced.

Even though the latter is more intuitive, and at least in my view,

more helpful for first-time learners, even if it is specific to one situation.

So the mathematician's answer to what are vectors is to just ignore the question.

In the modern theory, the form that vectors take doesn't really matter.

Arrows, lists of numbers, functions, pi creatures, really, it can be anything,

so long as there's some notion of adding and scaling vectors that follows these rules.

It's like asking what the number 3 really is.

Whenever it comes up concretely, it's in the context of some triplet of things,

but in math, it's treated as an abstraction for all possible triplets of things,

and lets you reason about all possible triplets using a single idea.

Same goes with vectors, which have many embodiments,

but math abstracts them all into a single, intangible notion of a vector space.

But, as anyone watching this series knows, I think it's better

to begin reasoning about vectors in a concrete, visualizable setting,

like 2D space, with arrows rooted at the origin.

But as you learn more linear algebra, know that these tools apply much more generally,

and that this is the underlying reason why textbooks and lectures tend to be phrased,

well, abstractly.

So with that, folks, I think I'll call it an in to this essence of linear algebra series.

If you've watched and understood the videos, I really do believe that

you have a solid foundation in the underlying intuitions of linear algebra.

This is not the same thing as learning the full topic, of course,

that's something that can only really come from working through problems,

but the learning you do moving forward could be substantially more efficient if you have

all the right intuitions in place.

So, have fun applying those intuitions, and best of luck with your future learning.