Orthogonal Projections on Inner Product Subspaces | Linear Algebra

Summary

This video explores the concept of orthogonal projections in linear algebra, starting with projecting a vector onto another vector and extending to projections onto subspaces. It introduces the projection theorem, explaining how any vector in an inner product space can be uniquely decomposed into a component within a subspace and one in its orthogonal complement. The video provides formulas for calculating projections using orthogonal or orthonormal bases and walks through a detailed example in \(\mathbb{R}^3\), illustrating both the projection and the orthogonal component. Visual explanations and practical calculations help clarify these essential concepts for understanding vector decomposition and projections in higher-dimensional spaces.

Takeaways

• 📐 The orthogonal projection of a vector U onto another vector V represents the component of U in the direction of V, often visualized as U's shadow on V.
• 🟢 A subspace spanned by scalar multiples of a vector V can be used to generalize projections from vectors onto lines to vectors onto subspaces.
• 🧩 The Projection Theorem states that any vector U in an inner product space V can be uniquely decomposed as U = W1 + W2, where W1 is in a subspace W and W2 is in the orthogonal complement of W.
• ⚡ The component of U orthogonal to a subspace W lies entirely within W's orthogonal complement, W⊥.
• ✏️ The projection of U onto a subspace W can be calculated if we have an orthogonal or orthonormal basis for W, using inner products with each basis vector.
• 🔢 For an orthogonal basis, the projection formula requires dividing by the squared norm of each basis vector, while for an orthonormal basis, the denominators are 1, simplifying calculations.
• 📊 Graphically in R³, projecting a vector onto a plane (subspace) separates it into a component on the plane and a component perpendicular to the plane.
• 🔍 The proof of the projection theorem uses linear combinations of orthogonal basis vectors and the property that vectors from the orthogonal complement yield zero inner products with basis vectors in W.
• 💡 Once the projection of U onto W is computed, the orthogonal component can be found by subtracting the projection from U: U⊥ = U − proj_W(U).
• 📏 An example in R³ demonstrates using orthonormal basis vectors to calculate the projection of a vector onto a plane and then deriving the component orthogonal to that plane.

Q & A

What is the orthogonal projection of a vector onto another vector?

The orthogonal projection of a vector u onto another vector v is the component of u that points in the direction of v. It can be visualized as the 'shadow' of u cast onto v.

How can we describe the projection of a vector onto a subspace?

The projection of a vector u onto a subspace W is the component of u that lies within W. It is obtained by decomposing u into a sum of a vector in W and a vector orthogonal to W.

What does the Projection Theorem state?

The Projection Theorem states that for a finite-dimensional subspace W of an inner product space V, every vector u in V can be uniquely expressed as u = W1 + W2, where W1 is in W and W2 is in the orthogonal complement of W (W⊥).

How do you compute the projection of a vector onto a subspace using an orthogonal basis?

Given an orthogonal basis {v1, ..., vr} for W, the projection of u onto W is calculated as proj_W(u) = Σ (⟨u, vi⟩ / ⟨vi, vi⟩) vi, summing over all basis vectors.

What simplification occurs when the basis is orthonormal?

If the basis vectors are orthonormal, each vector has unit norm, so the projection formula simplifies to proj_W(u) = Σ ⟨u, vi⟩ vi without needing to divide by the norm squared.

How do you find the component of a vector orthogonal to a subspace?

The component of a vector u orthogonal to W is found by subtracting the projection of u onto W from u itself: u - proj_W(u).

Why can we replace W1 with u in the inner product during the proof?

Because W2, the component orthogonal to W, is orthogonal to every basis vector in W, adding it to W1 in inner products does not change the values. Therefore, the inner product with u (which is W1 + W2) gives the same result.

In the R³ example, how is orthonormality of the basis vectors verified?

Orthonormality is verified by checking that the dot product of different basis vectors is zero (orthogonality) and that the norm of each basis vector is one (normalization).

What is the geometric interpretation of projecting a vector onto a subspace?

Geometrically, projecting a vector onto a subspace is like casting a shadow of the vector onto a plane or line representing the subspace, while the orthogonal component is perpendicular to that plane or line.

Why is the projection theorem useful in linear algebra?

The projection theorem is useful because it allows decomposition of vectors into components aligned with a subspace and its orthogonal complement, simplifying calculations, solving least squares problems, and providing geometric intuition.

What are the steps to calculate the projection of a vector onto a subspace with an orthonormal basis?

Step 1: Verify that the basis vectors are orthonormal. Step 2: Compute the inner product of the vector with each basis vector. Step 3: Multiply each basis vector by its corresponding inner product. Step 4: Sum these results to obtain the projection. Step 5: Subtract this projection from the original vector to get the orthogonal component.