Introduction to Linear Algebra

Abstract

Although the content in this chapter should be taught in much earlier life stages, graduate or undergraduate levels, of most possible reader, because this book mainly deals with somewhat data science oriented matters, it might not be a bad idea to reintroduce fundamental concepts in a data science oriented manner.

None can extinguish souls!

Momo Minamoto, Release the Spyce, Season 1, Episode 12

Appendix

1.1 I Rank

In this appendix, rank of matrix is briefly introduced because the concept of rank is important in the next chapter. Suppose that matrix $X \in \mathbb {R}^{N \times M}$ is represented as M N-dimensional vectors, $\boldsymbol {x}_j \in \mathbb {R}^N$, as

$$\displaystyle \begin{aligned} X = \left (\boldsymbol{x}_1, \ldots, \boldsymbol{x}_j, \ldots, \boldsymbol{x}_M \right). \end{aligned} $$

(1.46)

If there are vectors, $\boldsymbol {c}_j \in \mathbb {R}^{M'}, 1 \leq j \leq M$, such that

$$\displaystyle \begin{aligned} \boldsymbol{x}_j = \sum_{j'\in J} c_{jj'} \boldsymbol{x}_{j'} \end{aligned} $$

(1.47)

where J is a set of M′ integers taken from [1, M] without repetitions; the smallest M′ is called as the rank of matrix, otherwise the rank of matrix is equal to M. In other words, not all x _js are independent but at most M′ out of M x _js are independent. This means that x _js span not M dimensional space but at most M′(< M) dimensional space. Thus, the rank of tensor is at most min(M,N) because the number of independent vectors cannot exceed the number of dimensions.