3.6 Block matrices

A large matrix can contain a simple structure that is hard to see entry by entry. Its first few columns may describe one group of variables, its remaining columns another group, and its rows may come from several families of equations. A block partition records those groupings without changing any entry of the matrix.

Block notation is therefore a change of viewpoint, not a new kind of matrix. Every valid block calculation must still be justified by ordinary entrywise addition or ordinary row-by-column multiplication. The advantage is that a well-chosen partition exposes the bookkeeping before the arithmetic becomes large.

Choose cuts that expose structure

Suppose a linear model has one set of input variables of size $s_1$ and a second set of size $s_2$, while its outputs come in groups of sizes $r_1$ and $r_2$. A matrix representing the model naturally separates into four blocks. The block in position $(i,j)$ records how input group $j$ contributes to output group $i$.

This viewpoint is useful only when the cuts carry mathematical information. Typical choices isolate coefficient groups, identity or zero submatrices, repeated patterns, or successive stages of a linear transformation. An arbitrary grid is legal, but it may not simplify anything.

Three questions govern every block calculation:

1. What are the row heights and column widths of the displayed blocks?
2. Does the proposed operation make sense for the whole matrices?
3. If a block formula is used, do all of its block products and block sums have the required dimensions?

Keeping the second and third questions separate prevents a common error. Two matrices of the same outer size can always be added entry by entry, even when they are displayed with different cuts. What fails is only the attempt to pair their currently displayed blocks directly.

Record the sizes of the blocks

Definition

Block partition

Let $A \in \mathbb F^{m \times n}$. Choose positive integers

$$m=r_1+\cdots+r_p, \qquad n=s_1+\cdots+s_q.$$

Cut the rows into consecutive groups of heights $r_1,\ldots,r_p$ and the columns into consecutive groups of widths $s_1,\ldots,s_q$. Then

$$A=[A_{ij}]_{1\leq i\leq p,\,1\leq j\leq q}, \qquad A_{ij}\in\mathbb F^{r_i\times s_j}.$$

The submatrix $A_{ij}$ is the block in block row $i$ and block column $j$. Within one block row every block has the same number $r_i$ of rows, and within one block column every block has the same number $s_j$ of columns.

The displayed block grid is extra notation attached to the underlying matrix. Erasing every cut recovers the same $m \times n$ matrix. Adding a cut gives a refinement of the partition; erasing a cut gives a coarsening. If two partitions of the same outer rectangle differ, placing every cut from both partitions into one grid produces a common refinement.

Definition

Compatible partitions for a product

Let

$$A=[A_{ij}]_{p\times q}, \qquad B=[B_{jk}]_{q\times \ell}.$$

The partitions are compatible for multiplication when there are positive integers $r_i$, $s_j$, and $t_k$ such that

$$A_{ij}\in\mathbb F^{r_i\times s_j}, \qquad B_{jk}\in\mathbb F^{s_j\times t_k}.$$

Thus the column widths $s_j$ of $A$ agree, group by group, with the row heights $s_j$ of $B$. The number and positions of the inner cuts must agree; the outer row partition of $A$ and outer column partition of $B$ may be completely different.

Scalar multiplication needs no compatibility with another matrix: multiplying $A$ by a scalar simply multiplies every displayed block by that scalar.

Different block rows need not have equal heights, and different block columns need not have equal widths. Blocks need not be square, and a matrix need not have the same number of block rows and block columns. What matters is consistency along each full strip: one block-row height is shared across that row of blocks, and one block-column width is shared down that column of blocks. Writing these compositions beside a problem is often more reliable than judging compatibility from a drawn grid alone.

When block operations are valid

Theorem

Addition and transpose respect compatible block partitions

Let $A,B \in \mathbb F^{m\times n}$.

• The ordinary sum $A+B$ is defined whenever the outer dimensions agree, regardless of how $A$ and $B$ are currently displayed.
• If both matrices use the same row composition $(r_1,\ldots,r_p)$ and column composition $(s_1,\ldots,s_q)$, then

$$A+B=[A_{ij}+B_{ij}]_{p\times q}.$$

• If their displayed partitions differ, one may erase the cuts, repartition both matrices in the same way, or use the common refinement formed by the union of all row and column cut positions. Blockwise addition is then valid in that shared grid.
• If $A=[A_{ij}]_{p\times q}$ with $A_{ij}\in\mathbb F^{r_i\times s_j}$, then the block in position $(j,i)$ of the transpose satisfies

$$(A^T)_{ji}=A_{ij}^T, \qquad (A^T)_{ji}\in\mathbb F^{s_j\times r_i}.$$

Transpose exchanges block rows with block columns and transposes each block.

Theorem

Block multiplication theorem

Suppose

$$A=[A_{ij}]_{p\times q}, \qquad A_{ij}\in\mathbb F^{r_i\times s_j},$$

and

$$B=[B_{jk}]_{q\times \ell}, \qquad B_{jk}\in\mathbb F^{s_j\times t_k}.$$

Then the ordinary product $AB$ exists and has the block partition

$$AB=[C_{ik}]_{p\times \ell}, \qquad C_{ik}=(AB)_{ik}=\sum_{j=1}^{q}A_{ij}B_{jk}.$$

For every fixed $i,k$, each product $A_{ij}B_{jk}$ has size $r_i\times t_k$, so the terms in the sum really can be added. Consequently $C_{ik}\in\mathbb F^{r_i\times t_k}$.

For a $2\times2$ block grid the theorem reads

$$AB= \begin{bmatrix} A_{11}B_{11}+A_{12}B_{21} & A_{11}B_{12}+A_{12}B_{22}\\ A_{21}B_{11}+A_{22}B_{21} & A_{21}B_{12}+A_{22}B_{22} \end{bmatrix}.$$

The formula resembles multiplication of two numerical $2\times2$ matrices, but that resemblance is not permission to ignore sizes or order. Matrix block products generally do not commute, and every summand must have the same shape.

Why grouping entries preserves the rules

Proof

Why the block formulas are ordinary matrix formulas

For addition, fix any entry position $(a,b)$. The corresponding entry of $A+B$ is $a_{ab}+b_{ab}$. A shared block grid merely groups these entrywise sums into rectangles, so in block position $(i,j)$ the rectangle is exactly $A_{ij}+B_{ij}$. If the original cuts differ, their union still divides the same rows and columns into consecutive groups, which proves that a common refinement always permits direct block pairing.

For transpose, the entry in row $a$, column $b$ moves to row $b$, column $a$. Therefore the rectangle formerly in block position $(i,j)$ moves to position $(j,i)$, with its own rows and columns exchanged. This gives $A_{ij}^T$ and changes its size from $r_i\times s_j$ to $s_j\times r_i$.

For multiplication, fix a row belonging to block row $i$ of $A$ and a column belonging to block column $k$ of $B$. The ordinary dot product sums over every one of the shared inner coordinates. Split that single sum into the consecutive inner groups of sizes $s_1,\ldots,s_q$. The contribution from inner group $j$ is precisely the corresponding entry of $A_{ij}B_{jk}$. Adding the group contributions gives

$$(AB)_{ik}=A_{i1}B_{1k}+\cdots+A_{iq}B_{qk}.$$

Thus block multiplication does not introduce a new rule; it groups the terms of the usual row-by-column rule without omitting or duplicating any term.

The sequence below follows the same logic from partition choice to the assembled product.

See how cuts create block matrices, why blockwise addition needs the same partition, and how compatible block products reproduce the ordinary multiplication rule.

1. Partition first

   Horizontal and vertical cuts divide one matrix into rectangular blocks A_ij. If A_ij has m_i rows and p_j columns, the block labels carry size information.

2. Same cuts for addition

   Blockwise addition requires the same partition, so every A_ij has a corresponding B_ij of the same size.

3. Scalar reaches every block

   Scalar multiplication preserves the partition: cA is the block matrix whose blocks are cA_ij.

4. 2 x 2 block product

   For compatible 2 x 2 block partitions, the top-left block of AB is C_11=A_11B_11+A_12B_21, exactly mirroring ordinary matrix multiplication.

5. Compatibility check

   The formula works only if each A_ikB_kj is defined. If A_ik is m_i x p_k and B_kj is p_k x n_j, their product has size m_i x n_j.

6. Column-block bridge

   When B is partitioned into columns, AB=[Ab_1,Ab_2,...,Ab_r]. This is the same bookkeeping idea used later to organize several right-hand sides.

A block matrix is ordinary matrix algebra viewed at a larger scale. The partition must be fixed first, addition needs matching cuts, and multiplication is valid only when every block product in the formula is an ordinary defined matrix product.

From compatible shapes to a block product

Worked example

Read the symbolic 2 × 2 rule with dimensions

Let the row heights of $A$ be $(r_1,r_2)$, its column widths be $(s_1,s_2)$, and the column widths of $B$ be $(t_1,t_2)$. Then, for example,

$$A_{11}\in\mathbb F^{r_1\times s_1}, \quad B_{12}\in\mathbb F^{s_1\times t_2}, \quad A_{12}\in\mathbb F^{r_1\times s_2}, \quad B_{22}\in\mathbb F^{s_2\times t_2}.$$

Hence both $A_{11}B_{12}$ and $A_{12}B_{22}$ have size $r_1\times t_2$, and the upper-right block is

$$(AB)_{12}=A_{11}B_{12}+A_{12}B_{22} \in\mathbb F^{r_1\times t_2}.$$

This is the dimension check hidden inside the familiar symbolic formula. The same audit must succeed for every output block.

Worked example

Different displayed cuts do not prevent ordinary addition

Consider

$$A= \begin{bmatrix} 1&0&2&1\\ 3&1&-1&2\\ 0&4&5&-2 \end{bmatrix}, \qquad B= \begin{bmatrix} 2&1&0&3\\ -1&2&4&0\\ 5&0&1&1 \end{bmatrix}.$$

Suppose $A$ is displayed with row split $1+2$ and column split $2+2$, while $B$ is displayed with row split $2+1$ and column split $1+3$. Their current blocks cannot be paired: the upper-left block of $A$ is $1\times2$, whereas the upper-left block of $B$ is $2\times1$.

Nevertheless both underlying matrices are $3\times4$, so ordinary addition is defined:

$$A+B= \begin{bmatrix} 3&1&2&4\\ 2&3&3&2\\ 5&4&6&-1 \end{bmatrix}.$$

To recover a blockwise calculation, use all cut positions from both displays. The common row composition is $1+1+1$, and the common column composition is $1+1+2$. Every resulting small block of $A$ now has a corresponding block of the same size in $B$. The obstruction concerned the notation, not the matrix sum.

Worked example

Transpose with unequal rectangular blocks

Partition a $3\times5$ matrix using row composition $1+2$ and column composition $2+3$:

$$A= \left[ \begin{array}{cc|ccc} 1&2&0&3&4\\ \hline 5&6&7&8&9\\ 0&1&-1&2&3 \end{array} \right].$$

Thus the four blocks have sizes $1\times2$, $1\times3$, $2\times2$, and $2\times3$. Transposing gives

$$A^T= \left[ \begin{array}{c|cc} 1&5&0\\ 2&6&1\\ \hline 0&7&-1\\ 3&8&2\\ 4&9&3 \end{array} \right].$$

The new row composition is $2+3$ and the new column composition is $1+2$. In particular, the original upper-right $1\times3$ block becomes the lower-left $3\times1$ block after transposition. No equality of block heights and widths is required; the dimensions are exchanged.

Worked example

A complete numerical block product

Use row compositions $(2,1)$, inner composition $(1,2)$, and column compositions $(2,1)$:

$$A= \left[ \begin{array}{c|cc} 1&2&0\\ 3&-1&4\\ \hline 2&1&5 \end{array} \right], \qquad B= \left[ \begin{array}{cc|c} 1&2&3\\ \hline 0&1&2\\ 4&-1&1 \end{array} \right].$$

Here $A_{11}$ is $2\times1$, $A_{12}$ is $2\times2$, $B_{11}$ is $1\times2$, and $B_{21}$ is $2\times2$. The four output blocks are

$$\begin{aligned} C_{11} &=A_{11}B_{11}+A_{12}B_{21} =\begin{bmatrix}1&4\\19&1\end{bmatrix},\\ C_{12} &=A_{11}B_{12}+A_{12}B_{22} =\begin{bmatrix}7\\11\end{bmatrix},\\ C_{21} &=A_{21}B_{11}+A_{22}B_{21} =\begin{bmatrix}22&0\end{bmatrix},\\ C_{22} &=A_{21}B_{12}+A_{22}B_{22}=\begin{bmatrix}13\end{bmatrix}. \end{aligned}$$

For instance,

$$C_{12} =\begin{bmatrix}1\\3\end{bmatrix}[3] +\begin{bmatrix}2&0\\-1&4\end{bmatrix} \begin{bmatrix}2\\1\end{bmatrix} =\begin{bmatrix}3\\9\end{bmatrix} +\begin{bmatrix}4\\2\end{bmatrix} =\begin{bmatrix}7\\11\end{bmatrix}.$$

Assembling the blocks gives the ordinary product

$$AB= \begin{bmatrix} 1&4&7\\ 19&1&11\\ 22&0&13 \end{bmatrix}.$$

A dimension audit before multiplying blocks

Concept lensAlgebraic

Inner cuts group the same coordinates

Before carrying out arithmetic for an output block $(i,k)$, write the shapes:

$$(r_i\times s_j)(s_j\times t_k)=r_i\times t_k.$$

Do this for every inner index $j$. The repeated size $s_j$ first proves that each product exists. The unchanged outer sizes $r_i$ and $t_k$ then prove that the products can be added. Finally, the block-row heights of the answer sum to the number of rows of $A$, while its block-column widths sum to the number of columns of $B$.

For any proposed symbolic sum of matrix products, perform two checks: each product must exist, and all products being added must have the same output shape. In a genuine compatible block grid, the theorem's fixed $r_i$ and $t_k$ make the second check automatic once the block shapes have been recorded. If a loose collection of submatrices does not share those block-row heights and block-column widths, it is not a valid grid to which the theorem applies.

For a concrete partition choice, let $A$ be $2\times4$ and $B$ be $4\times3$. Suppose the columns of $A$ are grouped as $1+3$, but the rows of $B$ as $2+2$. Ordinary multiplication is defined; direct multiplication of those displayed blocks is not. The first proposed block product would have inner sizes $1$ and $2$, and the second would have inner sizes $3$ and $2$.

Keep both cut positions and refine the shared inner coordinates into groups of sizes $1+1+2$. Write the resulting blocks as

$$A=[C_1\ C_2\ C_3],\qquad B=\begin{bmatrix}D_1\\D_2\\D_3\end{bmatrix}.$$

Now $C_1,C_2$ are $2\times1$, $C_3$ is $2\times2$; $D_1,D_2$ are $1\times3$, and $D_3$ is $2\times3$. Therefore $AB=C_1D_1+C_2D_2+C_3D_3$, a sum of three $2\times3$ matrices. Refining the cuts repaired the block calculation without changing any matrix entry. The shared inner groups specify which coordinates contribute together; they are part of the justification for the formula.

Common mistakes

Common mistake

Confusing the matrix sum with a particular block display

Different displayed cuts do not make $A+B$ undefined when $A$ and $B$ have the same outer dimensions. They only prevent direct pairing of the current blocks. Erase the cuts, repartition both matrices alike, or take a common refinement.

Common mistake

Checking only the whole matrices

The fact that an $m\times n$ matrix can multiply an $n\times u$ matrix does not validate an arbitrary pair of block displays. The inner block boundaries must also agree. If they do not, repartition before applying a block formula.

Common mistake

Multiplying corresponding positions instead of block row by block column

The block $(AB)_{ik}$ is a sum over the inner index $j$; it is not merely $A_{ik}B_{ik}$. Read a complete block row of $A$ against a complete block column of $B$, in the given order.

Common mistake

Treating block products as commutative

Even when both $A_{ij}B_{jk}$ and $B_{jk}A_{ij}$ happen to exist, they need not be equal. The order in the block multiplication theorem is inherited from ordinary matrix multiplication and may not be reversed.

A checklist for block calculations

• A block partition is specified by row compositions and column compositions; every block in a block row has a common height, and every block in a block column has a common width.
• Equal outer dimensions are enough for ordinary matrix addition. Matching displayed partitions are required only for direct block-by-block pairing. Different partitions can be replaced by a common refinement.
• Transpose swaps the block grid and transposes each block: $(A^T)_{ji}=A_{ij}^T$.
• If $A_{ij}\in\mathbb F^{r_i\times s_j}$ and $B_{jk}\in\mathbb F^{s_j\times t_k}$, then $(AB)_{ik}=\sum_j A_{ij}B_{jk}$ has size $r_i\times t_k$.
• Block multiplication is ordinary row-by-column multiplication with its inner sum grouped into consecutive coordinate ranges. A dimension audit should precede the arithmetic.

Exercises

Checkpoint

Why do two block matrices need the same partition before you add them block by block?

Answer using the phrase "corresponding blocks."

Solution · Answer

Direct block-by-block addition needs the same partition because each block in one display must have a corresponding block of the same size in the other. This requirement concerns the chosen displays; the underlying matrices can still be added entry by entry whenever their outer dimensions agree.

Checkpoint

Find the size of $A_{11}B_{12}+A_{12}B_{22}$ when the four factors have sizes $2\times1$, $1\times4$, $2\times3$, and $3\times4$, respectively.

Check each product before checking whether they can be added.

Solution · Answer

Both products have size $2\times4$: their inner dimensions are $1$ and $3$, respectively. They can therefore be added, and the upper-right output block has size $2\times4$.

Checkpoint

Explain why block multiplication is still ordinary matrix multiplication in disguise.

Use the words "same rule" somewhere in your answer.

Solution · Guided solution

Block multiplication uses the same rule as ordinary multiplication. It groups the inner coordinates of each ordinary row-column dot product according to the chosen cuts; no term is added, removed, or reordered.

Checkpoint

Two $4\times4$ matrices have row partitions $1+3$ and $2+2$, and both have column partition $2+2$. Find the coarsest common row refinement that permits direct block addition.

Mark both row-cut positions within the same four-row array.

Solution · Guided solution

Retain the cuts after rows $1$ and $2$. The common row partition is $1+1+2$; the column partition stays $2+2$. Both matrices now have corresponding blocks of equal size, without changing any entry.

Checkpoint

Compute the upper-right block of a product from the given blocks.

Let

$$A_{11}=\begin{bmatrix}1&2\end{bmatrix}, \quad A_{12}=\begin{bmatrix}-1\end{bmatrix}, \quad B_{12}=\begin{bmatrix}1&0\\3&2\end{bmatrix}, \quad B_{22}=\begin{bmatrix}-2&1\end{bmatrix}.$$

Find $(AB)_{12}=A_{11}B_{12}+A_{12}B_{22}$ and state its size.

Solution · Guided solution

The first product has shape $(1\times2)(2\times2)=1\times2$; the second has shape $(1\times1)(1\times2)=1\times2$, so they can be added. Numerically,

$$A_{11}B_{12}=\begin{bmatrix}7&4\end{bmatrix}, \qquad A_{12}B_{22}=\begin{bmatrix}2&-1\end{bmatrix}.$$

Therefore

$$(AB)_{12}=\begin{bmatrix}9&3\end{bmatrix},$$

a $1\times2$ block. Writing the two shape calculations first explains why the numerical addition is legitimate.

Related notes

Review 3.2 Matrix multiplication, identity matrices, and linear systems if the ordinary multiplication pattern is still shaky.

For the next chapter, continue to 4.1 Homogeneous systems and null space. Later, 5.1 Invertible matrices connects matrix structure to reversibility.