Lec 1: Introduction to Machine Learning

Summary

Artificial Intelligence, Machine Learning, and Deep Learning

• Artificial intelligence (AI) refers to programs designed to learn and reason in ways associated with human intelligence.
• Machine learning (ML) is a subset of AI. Its algorithms learn from data rather than being explicitly programmed for every task. The course focuses mainly on statistical machine learning.
• Deep learning is a subset of machine learning based on advanced artificial neural networks. The lecturer says it addresses limitations of conventional neural networks and will be covered later in the course.

Pattern Recognition and Machine Perception

Pattern recognition uses machine-learning methods to identify patterns in data. A pattern can be an object, process, or event that can be named. A pattern class groups patterns that share attributes or come from the same source; a classifier assigns observed patterns to these classes.

Examples include:

• Speech and handwritten-character recognition
• Fingerprint and face recognition
• DNA-sequence identification
• Biomedical image and signal analysis
• Medical diagnosis, such as tumor detection
• Machine diagnostics and military image analysis

The lecturer compares machine perception with human perception: both take in sensory information, process it, and use the result to make a decision. In a computer-vision system, cameras acquire images or video, image-processing steps prepare the input, and algorithms make decisions such as recognizing an object.

Typical Pattern-Recognition Workflow

A pattern-recognition system generally follows these steps:

1. Acquire measurements: Use sensors, such as cameras or microphones, to collect data from patterns.
2. Preprocess the input: Improve data quality and prepare it for analysis. For images, this can include segmentation, which separates relevant foreground objects from the background.
3. Extract features: Convert the input into measurable properties that represent it.
4. Select features: Keep the features most useful for distinguishing the classes; not every extracted feature helps.
5. Represent the sample: Combine selected measurements into a feature vector, such as (X=(x_1,x_2,\ldots,x_d)).
6. Classify or make a decision: Apply a classifier to assign the feature vector to a class.
7. Evaluate and refine the system: Measure performance and use feedback to improve the system’s components or parameters.

Good features make classes easier to distinguish. They should be informative, ideally avoid unnecessary dependence on one another, and be robust to changes such as translation, rotation, scale, lighting, and noise. Poor or noisy features may make classification harder.

Classification Decisions and Decision Boundaries

A classifier partitions feature space into regions, with each region assigned to a class. The boundary between regions is a decision boundary; it may be linear or nonlinear.

• Hard decisions assign a sample to one class using a fixed boundary.
• Soft decisions, illustrated through fuzzy logic, allow a sample to have degrees of membership in multiple classes. A membership value between 0 and 1 expresses how strongly the sample belongs to a class.

A discriminant function assigns a score to a feature vector for each class. The classifier typically selects the class with the highest score. For two classes, the decision boundary occurs where their scores are equal. A linear discriminant can be written as (w^T x+b), where (w) is a weight vector and (b) is a bias.

Statistical Classification and Bayes’ Rule

In statistical pattern recognition, the goal is to estimate the probability that a sample belongs to a class given its features: the posterior probability (P(\omega_j\mid X)).

Bayes’ rule relates this probability to:

• The likelihood, or class-conditional density, (P(X\mid\omega_j))
• The prior probability of the class, (P(\omega_j))
• The evidence, which normalizes the probabilities

A classifier can choose the class with the larger posterior probability. The lecturer also introduces classification risk and error probability as considerations in making decisions.

Learning Approaches

• Supervised learning: Training examples have class labels. The system learns from labeled examples and then predicts labels for testing or new data. The lecturer mentions decision trees, random forests, k-nearest neighbors, logistic regression, naïve Bayes, and support vector machines.
• Unsupervised learning: Data are grouped according to similarity, often using a distance measure. This produces clusters without requiring class labels. The lecturer notes that clustering can be subjective: the same items may be grouped in different valid ways depending on the purpose.
• Semi-supervised learning: Uses a small amount of labeled data alongside a larger amount of unlabeled data. This can help when labels are difficult or costly to obtain, such as in medical-image classification.
• Reinforcement learning: An agent receives rewards or penalties for actions and learns toward a goal. The lecture briefly uses game playing as an example, while noting that reinforcement learning is not a main focus of the course.
• Regression: Fits a line or curve to data, for example, to find a best-fit relationship between sample points.

Feature Choice, Model Complexity, and Generalization

The fish-classification example illustrates why feature selection matters. Using only one feature, such as length, may cause seabass and salmon to be confused. Adding other features, such as lightness or width, can reduce errors if they are informative. Adding noisy or unhelpful features, however, may hurt performance.

The lecturer contrasts three model behaviors:

• Underfitting: A model that is too simple cannot represent the training data well. It has high error on both training and test data and is associated with high bias.
• Overfitting: A model that is too complex may fit the training samples extremely well but perform poorly on unseen data. This is associated with high variance.
• A reasonable compromise: A model should perform adequately on both training and test data rather than merely memorizing training examples.

The objective is to build a model that generalizes to novel data.

System Design and Evaluation

The design cycle described in the lecture is:

1. Collect data for training and testing.
2. Select useful, robust features.
3. Choose a model appropriate to the application.
4. Train the model.
5. Evaluate it on test samples, including its error rate.
6. Consider computational complexity, especially for real-time systems.
7. Use evaluation results to improve the overall system.

For classifier evaluation, the lecturer introduces the confusion-matrix outcomes: true positive, false negative, false positive, and true negative. These can be used to calculate measures such as accuracy, precision, recall, and specificity.

Classifier Families and Limits

• Generative classifiers model class-conditional data distributions and class priors. With parametric methods, the form of the distribution is assumed to be known, but its parameters must be estimated—for example, a Gaussian’s mean and covariance. Maximum-likelihood and Bayesian estimation are mentioned. Nonparametric methods estimate distributions without assuming a fixed form; Parzen windows and k-nearest neighbors are named.
• Discriminative classifiers focus on learning a decision boundary or directly distinguishing classes, rather than requiring an explicit model of each class’s density. Neural networks, perceptrons, and support vector machines are given as examples.
• Cover’s theorem, as introduced in the lecture, says that data that are not linearly separable in a low-dimensional space may become linearly separable after a suitable nonlinear mapping into a higher-dimensional space.
• The no-free-lunch principle cautions that no single classifier is best for every possible task; the appropriate method depends on the application.

Speakers and Sources

• Main speaker: The course lecturer, identified in the video metadata as Prof. M. K. Bhuyan of IIT Guwahati. The subtitles do not contain a clear spoken name.
• Source: NPTEL IIT Guwahati course lecture, Introduction to Machine Learning.
• Other audio: Brief music cues; no other identifiable speakers.