GCI World 2026 September Session 5 DuringLecture

Summary

The lecture introduces supervised learning, demonstrates three common algorithms in Python with scikit-learn, and explains how to evaluate and improve models.

Machine-learning foundations

• Artificial intelligence (AI) is the broad field of creating systems that perform tasks associated with human intelligence.
• Machine learning (ML) is an approach within AI in which programs learn patterns from data rather than relying only on explicit instructions.
• Deep learning is a type of machine learning based on neural networks.
• The lecture distinguishes three broad ML approaches:
  • Supervised learning: Learns from labeled examples with known answers.
  • Unsupervised learning: Looks for patterns in unlabeled data.
  • Reinforcement learning: Learns through actions, feedback, and rewards or penalties.
• In supervised learning, the model learns a relationship between explanatory variables (inputs) and a target variable (output). The goal is to make accurate predictions on data the model has not seen before—a property called generalization.
• Supervised tasks are generally either:
  • Regression: Predicts continuous values, such as a price or temperature.
  • Classification: Predicts discrete categories, such as “cat” or “dog.”

Three supervised-learning algorithms

• Linear regression predicts continuous values by fitting a line or plane to the data. It selects weights and a bias to minimize squared prediction errors.
• Logistic regression is used for classification despite its name. It applies a sigmoid function to produce probabilities between 0 and 1, then assigns a class using a threshold, often 0.5. It is trained by minimizing cross-entropy.
• Decision trees make predictions through a sequence of feature-based decisions. They can be used for both regression and classification. Their splits can be selected using measures such as information gain; tree depth and split criteria are examples of adjustable settings.

Common model-building workflow

The lecture presents this general process:

1. Prepare the data: Inspect it, clean it, handle missing values, and convert fields to appropriate numeric or categorical formats.
2. Choose the target and explanatory variables.
3. Split the data into training and test sets, for example, using scikit-learn’s train_test_split.
4. Initialize a model of the appropriate type.
5. Fit the model to the training data using fit.
6. Evaluate it on held-out data using score or suitable metrics.
7. Inspect and improve the model using techniques such as regularization, cross-validation, or hyperparameter tuning.
8. Use the final model to make predictions on new data.

In scikit-learn, the broad implementation pattern is similar across models: create an estimator, call fit, then use predict or score.

Practical examples

• Car-price regression: The lecture loads a car dataset, removes rows with missing values, converts string-formatted numeric fields, and predicts price from vehicle characteristics. It demonstrates splitting data into training and test sets and evaluating regression with R². Similar training and test R² values suggest the model performs comparably on both sets; a large gap can be a warning sign of overfitting.
• Income classification: A census-style dataset is used to predict whether annual income exceeds $50,000. The target is converted to a binary label, and logistic regression is evaluated by accuracy. The lecture cautions that accuracy can be misleading when classes are imbalanced: always predicting the majority class may already produce a seemingly high score. Other metrics—including AUC, precision, recall, and F1-score—may be more informative.
• Mushroom classification: Categorical features are converted with one-hot encoding, and a decision tree predicts whether mushrooms are poisonous. The tree can be visualized, and settings such as split criterion and maximum depth can be adjusted.
• In a live exercise, the instructor works through a car-price regression task, diagnoses an error caused by question-mark missing-value markers, and corrects the selection and preprocessing of the required columns before fitting and scoring the model.

Overfitting and regularization

• Overfitting occurs when a model fits the training data—including noise or incidental fluctuations—too closely, so it performs poorly on new data.
• Underfitting occurs when a model is too simple or insufficiently trained to capture important patterns.
• The aim is to find a balance: a model complex enough to learn useful relationships but not so complex that it fails to generalize.
• Possible ways to reduce overfitting include:
  • Regularization, which adds a complexity penalty to the loss function.
  • More training data.
  • Early stopping, which halts training before the model becomes overly fitted to its training data.
• Two regularization methods are described:
  • Ridge regression (L2): Penalizes the squared magnitudes of weights, encouraging smaller and more stable coefficients.
  • Lasso regression (L1): Penalizes the absolute magnitudes of weights and can shrink some coefficients to exactly zero, effectively selecting features.
  • Elastic Net: Combines L1 and L2 penalties.
• A practical comparison of ordinary linear regression and regularized models illustrates how their training and test scores can be compared. Regularization may improve generalization, though the size of the improvement depends on the data and settings.

Model evaluation and cross-validation

The lecture distinguishes three dataset roles:

• Training set: Used to learn model parameters.
• Validation set: Used during model selection and tuning.
• Test set: Held back for a final evaluation of the chosen model.

K-fold cross-validation provides a more dependable validation estimate than relying on just one train/validation split:

1. Divide the available data into K folds.
2. Train the model K times.
3. Each time, use a different fold for validation and the remaining folds for training.
4. Average the validation scores; the results can also be summarized with their standard deviation.

Stratified K-fold cross-validation is useful for classification with imbalanced classes. It aims to preserve the overall class proportions in each fold.

Hyperparameter tuning

• Hyperparameters are settings chosen before or during model selection rather than learned directly as ordinary model parameters. Examples include Ridge’s regularization strength and a decision tree’s maximum depth.
• Grid search evaluates combinations from a specified set of hyperparameter values and chooses the best-performing combination.
• In scikit-learn, GridSearchCV combines grid search with cross-validation. It can be configured with a model, a dictionary of candidate settings, and a number of folds, then fitted to the data.
• Grid search is systematic but can be computationally expensive, so the lecture recommends searching a reasonably limited set of candidates.

The overall lesson is to select a model suited to the task, validate it carefully, guard against overfitting, and tune its settings before relying on its performance on unseen data.

Speakers and sources featured

Speakers

• A prerecorded lecture narrator explains supervised learning concepts and the notebook demonstrations.
• A live instructor leads interactive notebook work, addresses a preprocessing error, and summarizes the practical exercises.
• No individual speaker names are provided in the subtitles.

Sources and materials

• Jupyter notebooks titled “Learning with a teacher” and “setup exercise.”
• scikit-learn, including its model classes, data-splitting and cross-validation utilities, and GridSearchCV.
• pandas, used for data loading and manipulation.
• Example datasets: car prices, census income, mushrooms, and breast cancer.
• Graphviz, mentioned for visualizing decision trees.