Let $T:V\to W$ be a linear transformation. Let $\alpha=\{v_1,\ldots,v_n\}$ be a basis for $V$, and let $\beta=\{w_1,\ldots,w_m\}$ be a basis for $W$. Let $\{e_1,\ldots,e_n\}$ and $\{f_1,\ldots,f_m\}$ be the standard basis for $\mathbb{R}^n$ and $\mathbb{R}^m$ respectively. A transformation matrix $[T]_\alpha^\beta$ of $T$ is a matrix such that the following diagram commutes:

where $\rho_V:v_i\mapsto e_i$ and $\rho_W:w_i\mapsto f_i$, i.e., we have the functional equation $\rho_W\circ T=[T]_{\alpha}^{\beta}\circ\rho_V$.

Example. Consider the linear transformation $T(x,y,z)=(x+y,y+z,z+x)$. Let $\{e_1,e_2,e_3\}$ be the standard bases on $\mathbb{R}^3.$ Let $\{w_1,w_2,w_3\}=\left\{\begin{bmatrix}1\\1\\0\end{bmatrix},\begin{bmatrix}0\\1\\1\end{bmatrix},\begin{bmatrix}1\\0\\1\end{bmatrix}\right\}$.

Then, the transformation matrix is

Example. Consider the differentiation operator $\frac{\partial}{\partial x}$ acting on polynomials of degree $n$. Let's find its transformation matrix. Consider the basis $\alpha=\{1,x,\ldots,x^n\}$ for $P_n(\mathbb{R})$ and the basis $\beta=\{1,x,\ldots,x^{n-1}\}$ for $P_{n-1}(\mathbb{R})$. We define the vertical maps $\rho_V$ by $x^i\mapsto e_{i+1}$ and $\rho_W$ by $x^i\mapsto f_{i+1}$, where $\{e_1,\ldots e_{n+1}\}$ is a basis for $\mathbb{R}^{n+1}$ and $\{f_1,\ldots,f_n\}$ is a basis for $\mathbb{R}^n$. Therefore, $\rho_V(a_0+a_1x+\cdots+a_nx^n)=a_0e_1+a_1e_1+\cdots+a_ne_{n+1}$ and $\rho_W\left(\frac{\partial}{\partial x}(a_0+a_1x+\cdots+a_nx^n)\right)=a_1f_1+2a_2f_2+\cdots+na_nf_n$. We can then write the transformation matrix

Let $A$ be an $m\times n$ matrix. We can treat $A$ as a linear transformation from $\mathbb{R}^n$ to $\mathbb{R}^m$. Let's explore the relationship between a matrix's column and row spaces and its corresponding linear transformation's range and null space.

The column space directly corresponds to the range of $A$. For any $y\in \text{Im}(A)$, we have $y=Ax$ for some $x\in \mathbb{R}^n$. Let $v_1,\ldots, v_n$ be the column vectors of $A$. If we write $x=\begin{bmatrix} x_1\\x_2\\\cdots\\x_n \end{bmatrix}$, then we can express $y$ as the column form of $Ax$,

Therefore, since any element of $\text{Im}(A)$ can be written as a linear combination of the column vectors of $A$, we can conclude that $\text{Im}(A)=C(A)$.

The following theorem will summarize the relationship between the column space and row space.

proof. The key insight is that elementary row operations preserve dimension since elementary row matrices are invertible. Let $E_n\cdots E_1A$ be a triangular system with elementary matrices $E_i$. This gives us $\text{rank}(E_n\cdots E_1A)=\text{rank}(A)$. Since $E_n\cdots E_1A$ is a triangular system, its nonzero row vectors form a basis for the row space of $A$. When we observe that the columns also form a triangular system, we can conclude that the dimension of the column space equals the number of nonzero row vectors. Thus, $\dim(C(E_n\cdots E_1A))=\dim(R(E_n\cdots E_1A))$.

Remark. Let $A:\mathbb{R}^n\to\mathbb{R}^m$ be an $m\times n$ matrix. Since $\dim(C(A))=\dim(R(A))=\text{rank}(A)$, we can use the rank-nullity theorem to show that a linear transformation induced by $A$ is not one-to-one (equivalently, the null space of $A$ is nontrivial) when $n>m$. The argument follows: first, we have

Therefore, $\text{nullity}(A)=n-m>0$ implies that the null space of $A$ is nontrivial and thus not one-to-one.

Finding a basis for the null space of $A$ is crucial because it helps determine the solution set of the linear equation $Ax=b$ when solutions exist. More precisely, we have the following equality:

where $x_0$ is a solution of $Ax=b$ and $N(A)$ is the null space of $A$. This equality can be proved through set inclusion in both directions.

($\subseteq$) If $x'$ is a solution, then $A(x'-x_0)=Ax'-Ax_0=b-b=0$. Therefore, $x'-x_0\in N(A)$.

($\supseteq$) An element $x'\in x_0+N(A)$ can be written as $x_0+n$ for some $n\in N(A)$. We compute $A(x_0+n)=Ax_0+An=b+0=b$.

We use the following example to show the method of finding a basis for a null space.

Example. Let us find all solutions for

This is equivalent to find a basis for the null space of the matrix

Applying Gaussian elimination to the end, we get

Now, here's a useful trick to complete our task. We'll label each column with a variable. If the $i$-th column has a pivot, then we can express that variable in terms of other variables. If the $i$-th column has no pivot, then the variable (let's call it $x_i$) is free to choose, and we'll add the trivial equation $x_i=x_i$ to our system of linear equations.

The last matrix above is the following linear equation system:

Therefore, we have

That is

Historically, determinants played a major role in the study of linear algebra. They served as a computational tool to determine whether a linear equation system was singular. Additionally, Cramer's rule used determinants to provide explicit formulas for solving linear equation systems. Today, however, we primarily use determinants for computing eigenvalues.

In this chapter, we will develop an intuitive method to define determinants. We can also provide an axiomatic definition for determinants. From this perspective, we can prove that the only function satisfying these axioms is the one we defined intuitively. We will present the axiomatic definition without developing the related theorems further. Although determinants are no longer central to mathematical research, some of their properties remain essential to know. We will cover these properties in workshops and homework assignments.

The goal is to find an invariant of square matrices that we can use to determine whether a linear equation $Ax=0$ has a unique solution. In $\mathbb{R}^2$, consider the system

has a unique solution if and only if the vectors $v_1=\begin{bmatrix}a_{11}\\a_{12}\end{bmatrix}$ and $v_2=\begin{bmatrix}a_{12}\\a_{21}\end{bmatrix}$ span a nondegenerate parallelogram, i.e., the area of the parallelogram spanned by $v_1$ and $v_2$ is nonzero. We use the following special case to illustrate the idea that the area is $a_{11}a_{22}-a_{12}a_{21}$.

The area of the parallelogram is $(a_{11}-a_{12})(a_{21}+a_{22})-(-a_{12}a_{22})-(a_{11}a_{21})=a_{11}a_{22}-a_{12}a_{21}$. This formula is generally true, and we left the details for the readers (see 3Blue1BrownThe determinant | Chapter 6, Essence of linear algebra).

This motivate us to define

Similarly, in $\mathbb{R}^3$, the system

has a unique solution if and only if the volume of the parallelepiped is nonzero. To find this volume, mathematicians expressed the parallelepiped spanned by the three column vectors in terms of $a_{ij}$. After carefully reorganizing the terms, the volume is given as

You can check the following two expressions are identical:

or

While there are three additional equivalent expressions, all six can be unified in the following definition.

We summarize our discussion in the following table.

Our goal is to extend the definition of determinant to $n\times n$ matrices where $n>3$. This definition must preserve the key property that a matrix $A$ has a nonzero determinant if and only if $Ax=0$ has a unique solution.

Let's examine the patterns we found for $n=2$ and $n=3$, and see if we can extend them to $n>3$.