Onlinemathtraining.com Website Review

Let’s look at several techniques in machine learning and the math topics that are used in the process. In linear regression, we try to find the best fit line or hyperplane for a given set of data points.  We model the output of our linear function by a linear combination of the input variables using a set of parameters as weights. The parameters are found by minimizing the residual sum of squares.  We find a critical point by setting the vector of derivatives of the residual sum of squares to the zero vector.  By the second derivative test, if the Hessian of the residual sum of squares at a critical point is positive definite, then the residual sum of squares has a local minimum there. In the above process, we used derivatives, the second derivative test, and the Hessian, which are notions from multivariable calculus.  We can also find the solution to our minimization problem using linear algebra. Let X be the matrix where the rows are our data inputs, beginning with 1 in each row, and y be the vector of our data outputs.  We want a vector β such that Xβ is close to y.  In other words, we want a vector β such that the distance ‖Xβ-y‖ between Xβ and y is minimized.  The vector β which minimizes the distance is such that Xβ is the projection of y onto the column space of X.  This is so because the projection of y onto the column space of X is the vector in the column space of X that is closest to y.  We then use the fact that Euclidean N-space can be broken into two subspaces, the column space of X and the orthogonal complement of the column space of X, and the fact that any vector in Euclidean N-space can be written uniquely as the sum of vectors in the column space of X and in the orthogonal complement of the column space of X, respectively, to deduce that y-Xβ is orthogonal to the columns of X.  From here, we can arrive at the matrix equation XT Xβ=XT y.  If XT X is positive definite, then the eigenvalues of XT X are all positive.  So 0 is not an eigenvalue