Lecture 01: Vectors in Machine Learning

The lecture begins by reintroducing vectors, a fundamental concept in machine learning and mathematics. Vectors are explained as mathematical objects that have length and direction, and exist in a vector space where they share common properties. They can be represented as 1-dimensional arrays, either as row or column vectors, in n-dimensional space. The lecturer emphasizes understanding the geometrical and mathematical representation of vectors, which is crucial for grasping machine learning algorithms.

In this section, the lecturer explains the representation of vectors in geometric models, touching on vectors in two-dimensional and three-dimensional space. Further, the basic operations of vector algebra like addition and subtraction are covered, illustrating with examples how these operations work dimensionally. The speaker discusses how vectors retain their properties in operations, adhering to vector space rules.

The lecture delves into dot products and how they result in scalar quantities. The concept is illustrated with examples, clarifying how the operation is performed in vector algebra. The lecturer proceeds to explain the concept of vector magnitude or length, which is obtained through the dot product of a vector with itself, followed by calculating the square root. This understanding is foundational for deeper vector algebra and machine learning.

Focus shifts to more complex operations such as calculating the angle between two vectors using their dot product and magnitudes. Then, the lecturer introduces linear combinations of vectors, explaining how new vectors can be formed through scalar multiplications and additions. Understanding linear combinations is stressed as pivotal for forming complex models and algorithms in machine learning.

The lecture explores the concepts of linear independence and dependence of vectors, including how to identify them through vector equations. Examples demonstrate these principles, showing how vectors can or cannot be represented in terms of others. This section lays the foundation for recognizing vector relationships that are crucial for algorithm development in machine learning.

Orthogonal and orthonormal vectors are introduced, detailing how orthogonal vectors have a dot product of zero and orthonormal vectors also have a length of one. The lecturer notes the importance of orthonality implying linear independence, a key concept later used in machine learning. Examples illustrate these principles, and the importance of converting linearly independent vectors to orthogonal vectors is outlined.