The Column Space & Column Rank of a Matrix

E. L. Lady

Let A = [ \begin{matrix}0 & 1 & 2 & −3 & 4 \\ 0 & 2 & 4 & 0 & 20 \\ 0 & −1 & −2 & 5 & 0 \end{matrix} ].

The column space of A is the subspace of ℝ^{3} spanned by the columns of A, in other words it consists of all linear combinations of the columns of A:

u [ \begin{matrix}0 \\ 0 \\ 0 \end{matrix} ] + w [ \begin{matrix}1 \\ 2 \\ −1 \end{matrix} ] + x [ \begin{matrix}2 \\ 4 \\ −2 \end{matrix} ] + y [ \begin{matrix}−3 \\ 0 \\ 5 \end{matrix} ] + z [ \begin{matrix}4 \\ 20 \\ 0 \end{matrix} ] .

It can then be seen that a vector b with coordinates b_{1} , b_{2} and b_{3} belongs to the column space of A precisely when the system of equations

[ \begin{matrix}0 & 1 & 2 & −3 & 4 \\ 0 & 2 & 4 & 0 & 20 \\ 0 & −1 & −2 & 5 & 0 \end{matrix} ] [ \begin{matrix}u \\ w \\ x \\ y \\ z \end{matrix} ] = [ \begin{matrix}b_{1} \\ b_{2} \\ b_{3} \end{matrix} ] = b

has a solution.

Now if we take the augmented matrix for this linear system and reduce it to row echelon form then we get

[ \begin{matrix}0 & 1 & 2 & −3 & 4 & | & b′_{1} \\ 0 & 0 & 0 & 1 & 2 & | & b′_{2} \\ 0 & 0 & 0 & 0 & 0 & | & b′_{3} \end{matrix} ]

(where b′_{1} , b′_{2} , and b′_{3} are new constants).

For instance, if we are trying to solve

[ \begin{matrix}0 & 1 & 2 & −3 & 4 \\ 0 & 2 & 4 & 0 & 20 \\ 0 & −1 & −2 & 5 & 0 \end{matrix} ] [ \begin{matrix}u \\ w \\ x \\ y \\ z \end{matrix} ] = [ \begin{matrix}3 \\ −6 \\ −7 \end{matrix} ]

then the augmented matrix reduces to the row echelon form

[ \begin{matrix}0 & 1 & 2 & −3 & 4 & | & 3 \\ 0 & 0 & 0 & 1 & 2 & | & −2 \\ 0 & 0 & 0 & 0 & 0 & | & 0 \end{matrix} ]

and so the solutions have the form

[ \begin{matrix}u \\ w \\ x \\ y \\ z \end{matrix} ] = [ \begin{matrix}0 \\ −3 \\ 0 \\ −2 \\ 0 \end{matrix} ] + r [ \begin{matrix}1 \\ 0 \\ 0 \\ 0 \\ 0 \end{matrix} ] + s [ \begin{matrix}0 \\ −2 \\ 1 \\ 0 \\ 0 \end{matrix} ] + t [ \begin{matrix}0 \\ −10 \\ 0 \\ −2 \\ 1 \end{matrix} ]

where r, s, and t are arbitrary constants.

Important: For instance, we can get one solution (a particular solution) by setting r = s = t = 0. In general, if a solution exists at all, then there is a solution in which all variables are zero which correspond to columns which do NOT contain leading entries in the row echelon form. (Be sure that you understand why this is true!)

By choosing the solution with r = s = t = 0 we get

[ \begin{matrix}3 \\ −6 \\ −7 \end{matrix} ] = 0 [ \begin{matrix}0 \\ 0 \\ 0 \end{matrix} ] − 3 [ \begin{matrix}1 \\ 2 \\ −1 \end{matrix} ] + 0 [ \begin{matrix}2 \\ 4 \\ −2 \end{matrix} ] − 2 [ \begin{matrix}−3 \\ 0 \\ 5 \end{matrix} ] + 0 [ \begin{matrix}4 \\ 20 \\ 0 \end{matrix} ] .

Applying this in general, we can see that if [ \begin{matrix}b_{1} \\ b_{2} \\ b_{3} \end{matrix} ] is in the column space of A,

then it is possible to write

[ \begin{matrix}b_{1} \\ b_{2} \\ b_{3} \end{matrix} ] = 0 [ \begin{matrix}0 \\ 0 \\ 0 \end{matrix} ] + w [ \begin{matrix}1 \\ 2 \\ −1 \end{matrix} ] + 0 [ \begin{matrix}2 \\ 4 \\ −2 \end{matrix} ] + y [ \begin{matrix}−3 \\ 0 \\ 5 \end{matrix} ] + 0 [ \begin{matrix}4 \\ 20 \\ 0 \end{matrix} ]

for the right choice of w and y, so that [ \begin{matrix}b_{1} \\ b_{2} \\ b_{3} \end{matrix} ] can be written as a linear

combination of the two columns [ \begin{matrix}1 \\ 2 \\ −1 \end{matrix} ] and [ \begin{matrix}−3 \\ 0 \\ 5 \end{matrix} ].

Restated, this says that

The column space of A is spanned by the two columns

[ \begin{matrix}1 \\ 2 \\ −1 \end{matrix} ] and [ \begin{matrix}−3 \\ 0 \\ 5 \end{matrix} ].

Since it can be easily (!) seen that these two vectors are linearly independent, it follows that they are a basis for the column space of A. Therefore the column rank of A, which by definition is the dimension of the column space, is 2.

As a general principle, let A by any m × n matrix and let A_{1}, …, A_{n} be the columns of A, so that A = [ A_{1} A_{2} … A_{n}]. Then these n columns A_{i} are vectors in ℝ^{m} and the column space of A consists of all vectors b ∈ ℝ^{m} which can be written in the form

b = x_{1}A_{1} + ⋯ + x_{n}A_{n} = A [ x_{1} ; ⋮ ; x_{n} ] .

Now the procedure for solving systems of linear equations by elimination shows that if it is possible to solve the system

A [ x_{1} ; ⋮ ; x_{n} ] = b

at all, then there is one solution where only those variables x_{j_{1}}, …, x_{j_{r}} corresponding to columns containing the leading entry for some row in the row echelon form of A need to be non-zero. (So here r will be the number of leading entries in the row echelon form, which is the same as the number of non-zero rows in the row echelon form.) This says that if b is in the column space of A then there exist values x_{j_{1}}, …, x_{j_{r}} such that b = x_{j_{1}}A_{j_{1}} + ⋯ + x_{j_{r}}A_{j_{r}}.

Restated, this says that the column space of A is spanned by the columns A_{j_{1}}, …, A_{j_{r}}, where j_{1}, …, j_{r} are the numbers for the columns which contain the leading entry for some row in the row echelon form of A. (Thus r is the number of non-zero rows in the row echelon form of A.)

Furthermore, these columns A_{j_{1}}, …, A_{j_{r}} will always be linearly independent. In fact, we can see that x_{j_{1}}A_{j_{1}} + ⋯ + x_{j_{r}}A_{j_{r}} = 0 is only possible when x_{j_{1}} = ⋯ = x_{j_{r}} = 0 by noticing that the row echelon form of [A_{j_{1}} … A_{j_{r}}] is an m × r matrix whose row echelon form has a leading entry in each column. (Why? Hint: How does the row echlon form for [A_{j_{1}} … A_{j_{r}}] compare to the row echelon form of A = [A_{1} … A_{n}]?)