Linear Algebra for Machine Learning

A vector is an ordered collection of components; a matrix arranges components in rows and columns. In a common data convention, rows are examples and columns are features. Record that convention explicitly. A 100-by-3 data matrix X and a 3-by-1 coefficient vector w produce a 100-by-1 result Xw: one linear score for each row. Other conventions are possible, so dimensions and documentation must agree. A dot product multiplies matching components and adds the products. For [1, 2] and [2, −1], it is 1 × 2 + 2 × (−1) = 0. Matrix-vector multiplication applies that operation to each matrix row. With X containing rows [1, 2] and [3, 4], and w = [2, −1], the result is [0, 2]. This calculation gives scores, not automatically probabilities or correct classifications. Keep matrix multiplication distinct from multiplying matching entries. A transpose swaps rows and columns. For matrices A and B, AB and BA can have different dimensions, and one may be undefined; even when both exist, they need not be equal. Write the intended operation before choosing a programming operator. Rank describes the number of independent columns or rows. Identical feature columns do not supply two independent directions, and a square matrix is invertible only when it has full rank. Learn linear systems, orthogonality and projections through small examples before moving to eigenvectors or singular value decomposition. These ideas support least-squares fitting and dimensionality reduction, but an elegant matrix expression does not establish that a dataset is suitable. Check feature definitions, units and ordering alongside the algebra.