Lecture 24: Positive Definite Matrix And Quadratic Forms of Matrices|GATE DA Linear Algebra Module-2

Summary

The lecture introduces positive definite matrices, positive semidefinite matrices, and quadratic forms, explaining how to test for these properties and how they connect to optimization.

Positive definite matrices

A real matrix (A) is positive definite (PD) if it is symmetric and all its eigenvalues are positive. Because a symmetric matrix has real eigenvalues, they can be classified by sign.

Finding eigenvalues directly can be difficult for large matrices, so the lecture presents alternative tests. Positive definite matrices are important in multivariable optimization, especially when studying conditions for minima and maxima.

Tests for positive definiteness

For a real symmetric matrix, the lecture discusses these tests:

1. Eigenvalue test: All eigenvalues are (>0).
2. Pivot test: Under suitable symmetric elimination, the pivots are positive.
3. Leading principal minors test: The determinants of the upper-left (1\times1, 2\times2,\ldots,n\times n) submatrices are all (>0).
4. Quadratic-form test: [ x^{T}Ax>0 \quad \text{for every nonzero vector }x. ]

Any one valid test is sufficient; it is not necessary to apply all of them.

The lecture illustrates the quadratic-form test by expanding (x^{T}Ax) and rewriting it as a sum of squares. If the expression is positive for every nonzero (x), the matrix is positive definite.

Ordinary row operations generally do not preserve eigenvalues. Therefore, row-reducing a matrix and reading its diagonal entries as its eigenvalues is not valid.

Properties and cautions

• If (A) is positive definite, then (A^{-1}) is also positive definite: its eigenvalues are the reciprocals of those of (A), and remain positive.

• If (A) and (B) are positive definite, then (A+B) is positive definite. This follows from [ x^{T}(A+B)x=x^{T}Ax+x^{T}Bx. ]

• The eigenvalues of (A+B) are not generally obtained by simply adding corresponding eigenvalues of (A) and (B), since their eigenvectors may differ.

Positive semidefinite matrices

A symmetric matrix is positive semidefinite (PSD) if all its eigenvalues are (\geq 0). Equivalently, [ x^{T}Ax\geq 0 \quad \text{for every }x. ]

A PSD matrix may have zero eigenvalues; a positive definite matrix may not. The lecture gives examples and discusses determining a parameter range for which a matrix is PSD.

Technical clarification: For positive definiteness, the test that all leading principal minors are positive is valid. For positive semidefiniteness, merely checking that the leading principal minors are nonnegative is not sufficient in general. The standard principal-minor test requires all principal minors to be nonnegative. Pivot-based PSD tests also require care when zero pivots occur.

Negative definite and negative semidefinite matrices

The lecture briefly introduces the analogous terms:

• Negative definite: (x^{T}Ax<0) for every nonzero (x), equivalently all eigenvalues are negative.
• Negative semidefinite: (x^{T}Ax\leq0) for every (x), equivalently all eigenvalues are nonpositive.

A symmetric matrix need not belong to any of these four categories; it can have both positive and negative eigenvalues.

Quadratic forms and their matrix representation

The subtitles repeatedly say “quartic form,” but the topic being described is a quadratic form: a scalar-valued expression involving squared variables and products such as (x_1x_2).

A quadratic form can be written as [ q(x)=x^{T}Ax ] for a symmetric matrix (A).

• The diagonal entries of (A) are the coefficients of the squared terms.
• For a cross term such as (c x_1x_2), the symmetric matrix places (c/2) in each of the two corresponding off-diagonal positions. Their combined contribution is (c x_1x_2).
• A nonsymmetric matrix can sometimes represent the same quadratic form, but the symmetric representation is the standard, uniquely determined one.

The lecture also shows how to evaluate a quadratic form by substituting a given vector or by multiplying (x^{T}Ax) directly.

Diagonalizing a quadratic form

For a real symmetric matrix, the spectral decomposition gives [ A=PDP^{T}, ] where (P) is orthogonal and (D) is diagonal with the eigenvalues of (A).

Setting (y=P^{T}x) rewrites the quadratic form as [ x^{T}Ax=y^{T}Dy. ]

This change of coordinates removes cross terms and expresses the form using the eigenvalues. The lecture presents this as a useful change-of-variable idea, while noting that its main practical focus is recognizing and representing quadratic forms.

Speakers and sources featured

• Speaker: An unnamed instructor from GO Classes.
• Audience participants: Students ask questions or respond; some names in the auto-generated subtitles are unclear.
• Sources mentioned: A text or author referred to as “David C.” (the full reference is unclear in the subtitles); examples attributed to IIT Madras and the University of Utah.