5.1 Invertible matrices

Invertibility is one of the first places where linear algebra becomes more than a procedure for solving one system. A square matrix is invertible exactly when it can be undone by another square matrix, and that idea turns out to be equivalent to many other statements: row reduction to the identity, consistency of every system $Ax = b$ , linear independence of the columns, and the ability to write every vector as a linear combination of those columns.

This note develops those equivalences carefully. The goal is not only to know what the symbol $A^{-1}$ means, but also to recognize when it exists and how to use it without guessing.

Left and right inverses

Before the square case, it is useful to separate two one-sided notions.

Definition

Left inverse and right inverse

Let $A$ be a $p \times q$ matrix.

A $q \times p$ matrix $H$ is a left inverse of $A$ if $HA = I_q$ .
A $q \times p$ matrix $G$ is a right inverse of $A$ if $AG = I_p$ .

These definitions matter because matrix multiplication is not commutative. For rectangular matrices, a left inverse and a right inverse need not both exist. The square case is special.

Definition

Invertible matrix

Let $A$ be a $p \times p$ square matrix. We say that $A$ is invertible if there exists a $p \times p$ matrix $B$ such that

BA = AB = I_p.

The matrix $B$ is called the inverse of $A$ , and we write $B = A^{-1}$ .

Theorem

The inverse is unique

If $B$ is a left inverse of $A$ and $C$ is a right inverse of $A$ , then $B = C$ . So an invertible matrix has exactly one inverse.

Proof

Why the inverse is unique

What invertibility means

Invertibility is a reversibility statement. Applying $A$ changes a vector, but if $A$ is invertible then $A^{-1}$ undoes that change exactly.

That is why the identity matrix appears in the definition. The identity matrix does nothing:

I_p x = x

for every compatible vector x. An inverse is precisely a matrix that brings you back to that unchanged state.

Worked example

A diagonal matrix is easy to invert

Let

D = \operatorname{diag}(2, -1, 3).

Then

D^{-1} = \operatorname{diag}\left(\tfrac{1}{2}, -1, \tfrac{1}{3}\right).

This works because each diagonal entry is replaced by its reciprocal, and the off-diagonal zeros stay zero. Multiplying $D$ by $D^{-1}$ gives $I_3$ .

Row reduction and the inverse

The most practical way to test invertibility is to row-reduce. The key point has two parts:

row-operation matrices are invertible, with inverse given by the reverse row operation;
a square matrix is invertible exactly when it can be row-reduced to $I_p$ .

Theorem

Row-operation matrices are invertible

If $\rho$ is a row operation on matrices with p rows, and $\bar{\rho}$ is the reverse row operation, then the corresponding row-operation matrices $M[\rho]$ and $M[\bar{\rho}]$ satisfy

M[\rho]^{-1} = M[\bar{\rho}], \qquad M[\bar{\rho}]^{-1} = M[\rho].

This gives a clean interpretation of row reduction: every row operation is actually multiplication on the left by an invertible matrix.

Theorem

Invertibility and row reduction

For a square matrix $A$ , the following are equivalent:

$A$ is invertible.
$A$ is row-equivalent to $I_p$ .
$A$ is a product of $p \times p$ row-operation matrices.
$A$ is nonsingular.

The practical consequence is very concrete: if row operations transform $A$ to $I_p$ , then those same operations, applied to $[A | I_p]$ , transform it to $[I_p | A^{-1}]$ .

Read and try

Follow one inverse-by-row-reduction example

The live demo lets you step through [A | I] until the left block becomes I.

Start from [A | I]. If A is invertible, row reduction will turn the left block into I.

1	2	1	1	0	0
0	1	1	0	1	0
2	3	4	0	0	1

The live demo above is the shortest way to see the logic. It is not the definition. It is the computational method that matches the definition.

Read and try

Trace one full row-reduction path

The live stepper walks through one complete elimination path, showing the row operation, the pivot you are focusing on, and the matrix produced at each step.

1	2	2	4
1	3	3	5
2	6	5	6

Row operation

Choose the first pivot in column 1.

What to notice

Column 1 already has a convenient pivot 1 in the first row, so we do not need a row swap.

Start with the augmented matrix. The first pivot should help us clear the entries underneath it.

The second widget shows the shape of a full elimination path. In an invertible case, the left block eventually becomes $I_p$ , and that is the moment when the right block becomes the inverse.

Equivalent formulations

Invertibility is useful because it has a dictionary of equivalent conditions. This is the main bridge between algebra, row reduction, and systems of linear equations.

Theorem

Equivalent ways to recognize invertibility

Let $A$ be a $p \times p$ matrix. Then the following statements are equivalent:

$A$ is invertible.
$A$ is row-equivalent to $I_p$ .
$A$ is a product of row-operation matrices.
$A$ has a left inverse.
$A$ has a right inverse.
$A$ is nonsingular.
For every column vector b with p entries, the system $Ax = b$ is consistent.
For every column vector b with p entries, the system $Ax = b$ has the unique solution $x = A^{-1}b$ .

Two of these statements are especially important in practice.

Statement 7 says that the columns of $A$ span $R^p$ .
Statement 8 says that invertibility gives you a complete solution formula, not just existence.

That is why invertibility is the exact algebraic condition behind solving a linear system by a matrix inverse.

Row-equivalence through invertible matrices

We can push the row-operation viewpoint one step further. Instead of thinking about row-equivalence as a long list of elementary moves, package the whole list into one invertible matrix on the left.

Theorem

Row-equivalence is left multiplication by an invertible matrix

Suppose $A$ and $B$ are matrices with p rows. Then the following are equivalent:

$A$ and $B$ are row-equivalent.
There exists an invertible $p \times p$ matrix $G$ such that

B = GA.

Moreover, once $B = GA$ , we also have

A = G^{-1}B.

This theorem is not a new computational trick. It is a cleaner language for the same phenomenon. A sequence of row operations can always be compressed into one invertible matrix $G$ , and the reverse row operations are encoded by $G^{-1}$ .

Worked example

Reading a row-equivalence as one matrix equality

Let

A = \begin{bmatrix} 1 & 0 & 1 \\ 1 & 2 & 3 \\ 0 & 1 & 1 \end{bmatrix}, \qquad G = \begin{bmatrix} 1 & 0 & 0 \\ -1 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix}.

The matrix $G$ is the row-operation matrix for the move

R_2 \leftarrow R_2 - R_1.

GA = \begin{bmatrix} 1 & 0 & 1 \\ 0 & 2 & 2 \\ 0 & 1 & 1 \end{bmatrix}.

If we call this new matrix $B$ , then $B = GA$ . That single equation records the entire row operation. Since $G$ is invertible, $A$ and $B$ are row-equivalent.

The gain is conceptual. Once you know that row-equivalence means multiplication by an invertible matrix on the left, you can explain many invariants in one line instead of by repeating row-operation arguments.

Theorem

Row operations preserve linear relations among corresponding columns

Let $A$ and $B$ be row-equivalent $p \times q$ matrices, and write their columns as

A = [a_1 \; a_2 \; \cdots \; a_q], \qquad B = [b_1 \; b_2 \; \cdots \; b_q].

a_j = \alpha_1 a_{k_1} + \alpha_2 a_{k_2} + \cdots + \alpha_n a_{k_n},

then

b_j = \alpha_1 b_{k_1} + \alpha_2 b_{k_2} + \cdots + \alpha_n b_{k_n}.

In particular, linear dependence and linear independence among corresponding columns are preserved by row-equivalence.

The proof is short once $B = GA$ is known. Multiply the relation for the columns of $A$ by $G$ . Because matrix multiplication is linear,

Ga_j = \alpha_1 G a_{k_1} + \alpha_2 G a_{k_2} + \cdots + \alpha_n G a_{k_n},

which is exactly the corresponding relation amongst the columns of $B$ .

This is the bridge from row reduction to column language. Row operations change the actual columns, but they do not change which columns are redundant or which column relations are forced by the others.

Why the reduced row-echelon form is unique

The reduced row-echelon form in a row-equivalence class is unique. That fact is easy to overlook, but it is what makes later definitions mathematically legitimate.

Theorem

A row-equivalence class has exactly one reduced row-echelon form

Suppose $A$ is a matrix, and suppose $B$ and $C$ are both reduced row-echelon forms. If $B$ is row-equivalent to $A$ and $C$ is row-equivalent to $A$ , then

B = C.

A standard proof uses induction on the rank. The basic strategy is:

compare the pivot columns from left to right,
use preserved linear relations to force the same pivot positions, and then
show that every free column must have the same coefficients in terms of the pivot columns.

So reduced row-echelon form is not merely a convenient final answer. It is the final answer inside a row-equivalence class.

Definition

Rank

The rank of a matrix is the number of pivots in its reduced row-echelon form.

This definition works only because the reduced row-echelon form is unique. If different reduction paths could produce different reduced forms with different numbers of pivots, then rank would depend on the calculation. The uniqueness theorem rules that out.

Column independence and linear combinations

The same source also recasts invertibility in terms of columns.

Theorem

Invertibility and the columns of a square matrix

For a $p \times p$ matrix $A$ , the following are equivalent:

$A$ is invertible.
The columns of $A$ are linearly independent.
Every column vector in $R^p$ is a linear combination of the columns of $A$ .

These are not separate facts. They are three ways of reading the same structural statement.

If the columns are linearly independent, then no column is redundant. If they span $R^p$ , then every target vector can be built from them. For a square matrix, those two conditions coincide exactly when the matrix is invertible.

Why the transpose also matters

Invertibility behaves well under transpose.

Theorem

Transpose and powers

If $A$ is invertible, then:

$A^t$ is invertible, and $(A^t)^{-1} = (A^{-1})^t$ .
$A^n$ is invertible for every integer n, and $(A^n)^{-1} = (A^{-1})^n$ .

The transpose result is useful when you want to turn a statement about columns into a statement about rows. The power rule is useful when a repeated transformation appears in a calculation.

Worked example

Find an inverse by row reduction

Let

A = \begin{bmatrix} 1 & 2 \\ 3 & 5 \end{bmatrix}.

Start from $[A | I_2]$ :

\left[\begin{array}{cc|cc} 1 & 2 & 1 & 0 \\ 3 & 5 & 0 & 1 \end{array}\right].

Eliminate the entry below the first pivot:

\left[\begin{array}{cc|cc} 1 & 2 & 1 & 0 \\ 0 & -1 & -3 & 1 \end{array}\right].

Now scale the second row and clear the entry above the second pivot:

\left[\begin{array}{cc|cc} 1 & 0 & -5 & 2 \\ 0 & 1 & 3 & -1 \end{array}\right].

A^{-1} = \begin{bmatrix} -5 & 2 \\ 3 & -1 \end{bmatrix}.

The computation is not the point by itself. The point is that the right block of the augmented matrix records the inverse because the left block has been driven to the identity.

Common mistakes

Common mistake

Do not confuse one-sided inverses in the rectangular case

For a non-square matrix, having a left inverse does not automatically mean it has a right inverse. The square case is special: once an inverse exists, it is both a left inverse and a right inverse, and it is unique.

Common mistake

Answer in one sentence using the phrase “well defined.”

Solution

Answer

Exercise

Quick check

Suppose $A$ is invertible and $AB = I_p$ . Prove that $B = A^{-1}$ .

Use the fact that the inverse of $A$ is unique.

Solution

Guided solution

Read this first

This page depends especially on 2.3 Gaussian elimination and RREF, 3.1 Matrix multiplication and identity matrices, and 3.2 Transpose and special matrices.

5.1 Invertible matrices

MATH1030: Linear algebra I

Left and right inverses

Left inverse and right inverse

Invertible matrix

The inverse is unique

Why the inverse is unique

What invertibility means

A diagonal matrix is easy to invert

Row reduction and the inverse

Row-operation matrices are invertible

Invertibility and row reduction

Follow one inverse-by-row-reduction example

Trace one full row-reduction path

Equivalent formulations

Equivalent ways to recognize invertibility

Row-equivalence through invertible matrices

Row-equivalence is left multiplication by an invertible matrix

Reading a row-equivalence as one matrix equality

Row operations preserve linear relations among corresponding columns

Why the reduced row-echelon form is unique

A row-equivalence class has exactly one reduced row-echelon form

Rank

Column independence and linear combinations

Invertibility and the columns of a square matrix

Why the transpose also matters

Transpose and powers

Worked example

Find an inverse by row reduction

Common mistakes

Do not confuse one-sided inverses in the rectangular case

Do not guess invertibility from appearance

Quick checks

If AAA is invertible, what is A−1AA^{-1}AA−1A?

Answer

If AAA is invertible, can the homogeneous system Ax=0Ax = 0Ax=0 have a nonzero solution?

Answer

If AAA is invertible, is AtA^tAt invertible?

Answer

If B=GAB = GAB=GA with GGG invertible and the columns of AAA satisfy a3=2a1−a2a_3 = 2a_1 - a_2a3​=2a1​−a2​, what relation must hold among the columns of BBB?

Answer

Why does uniqueness of RREF matter when defining rank?

Answer

Exercise

Suppose AAA is invertible and AB=IpAB = I_pAB=Ip​. Prove that B=A−1B = A^{-1}B=A−1.

Guided solution

Read this first

Section mastery checkpoint

Prerequisites

Key terms in this unit

More notes in this series

If $A$ is invertible, what is $A^{-1}A$ ?

If $A$ is invertible, can the homogeneous system $Ax = 0$ have a nonzero solution?

If $A$ is invertible, is $A^t$ invertible?

If $B = GA$ with $G$ invertible and the columns of $A$ satisfy $a_3 = 2a_1 - a_2$ , what relation must hold among the columns of $B$ ?

Suppose $A$ is invertible and $AB = I_p$ . Prove that $B = A^{-1}$ .