Lec 02 Fundamentals of Machine Learning

Main ideas and lessons (structured)

1) Big-picture categorization of Machine Learning

Machine learning is grouped into three main types (with minor variations possible, but not covered in detail):

• Supervised learning
• Unsupervised learning
• Reinforcement learning

A key goal of the lecture is to explain how these categories differ and how they determine what kinds of problems can be tackled in core engineering disciplines.

---

2) Supervised learning (with examples + regression vs classification)

Core concept

In supervised learning, a model is trained to predict a target variable using input features. Labels are required for every data point in the training set.

Training data provides, for each example:

• an input feature vector
• the corresponding target label/value

Target variable examples (domain contexts)

• Chemical/mechanical/aerospace engineering

  • Predict reactor temperature
  • Predict drag (and/or lift) experienced by a body moving through air

• Materials/battery/electrolyzer engineering

  • Predict material efficiency (e.g., for batteries/electrolyzers)

• Plant/safety applications

  • Predict reactor temperature to prevent accidents (e.g., runaway reactions / explosion risk)
  • Detect issues such as sensor failure or fault conditions

Regression vs classification (subclassification of supervised learning)

• Regression (continuous target)

  • Predict a continuous value
  • Example: reactor temperature (numeric)

• Classification (discrete target)

  • Predict membership in fixed categories
  • Example: overheated vs not overheated (binary classification)
  • Types:
    • Binary classification: 2 categories (yes/no)
    • Multiclass classification: more than 2 categories

Visualization-based distinction (feature space explanation)

Consider a feature space with two features (feature 1 and feature 2):

• Supervised learning

  • Training points are already colored/labeled (e.g., blue/red/green).
  • The algorithm learns boundaries/associations between feature values and known classes.
  • For a new point, the algorithm predicts its class (e.g., green).

• Unsupervised learning

  • The algorithm sees only the feature values (no class labels).
  • It discovers structure/patterns and clusters naturally (e.g., discovers red/blue/green groups).

Supervised learning use cases (labeled data required)

• Drug/material property prediction

  • Given molecular structure, predict whether it is soluble in water (yes/no)

• Battery performance

  • Given battery chemistry + operating conditions + history, predict:
    • voltage over time
    • and/or failure behavior

• Medical image + symptom inference

  • Given X-ray images and other symptoms/parameters, predict whether a person has chest congestion / a related condition (yes/no)

Common supervised algorithms mentioned

• Linear regression / nonlinear regression
• Decision trees
• Random forests
• Support vector machines (SVMs)
• Neural networks / deep learning

---

3) Unsupervised learning (no labels; structure discovery + subtypes)

Core concept

In unsupervised learning, there is no labeled target variable. The model receives only a dataset (many data points) and learns:

• underlying structure
• similarities/differences between points
• often using a similarity metric

The learned structure can support later tasks, including improved modeling/prediction.

Dimensionality reduction within unsupervised learning

A major unsupervised goal is to reduce the number of features by:

• combining features into a more condensed representation
• preserving important relationships as much as possible

This can make data easier to understand and can improve downstream supervised learning.

Unsupervised learning types and algorithm examples

• Clustering

  • Group data points based on similarity/difference
  • Examples:
    • K-means
    • density-based clustering (mentioned for discussion later)

• Dimensionality reduction

  • Examples:
    • PCA (linear dimensionality reduction)
    • autoencoders (nonlinear)
    • t-SNE / UMAP (nonlinear dimensionality reduction techniques)

• Generative algorithms

  • Learn the data distribution and generate new samples
  • Learn an underlying probability distribution to generate new data points similar to training data
  • Example motivation: drug discovery
    • replace/design alternative molecules similar to known effective ones
  • Algorithm examples:
    • GANs (Generative Adversarial Networks)
    • VAEs (Variational Autoencoders)
    • Restricted Boltzmann Machines
  • Intended outcome: propose replacement candidates (new molecules)

Unsupervised examples relevant to engineering

• Materials/chemistry

  • Categorize elements by learned similarity (using featurized periodic-table elements)
  • Aid in developing new catalysts/materials

• Molecular generation

  • Generate molecules similar to those in the training set
  • Useful in drug discovery/pharmaceutical engineering contexts

---

4) Reinforcement learning (briefly, for completeness)

Core concept

A virtual agent interacts with an environment:

• The agent takes actions that change the environment.
• After each action, the agent receives:
  • a reward (or) penalty
• The learning objective is to maximize cumulative reward
• This yields an optimal policy/strategy

Course scope note

Reinforcement learning is introduced conceptually, but the course will mainly focus on supervised and unsupervised learning.

---

Supervised learning methodology concepts (detailed bullet format)

A) How supervised learning is framed as a function approximator

Supervised ML aims to learn a model (denoted as f):

• Target variable: Y
• Features: x₁, x₂, …, xₙ

The model maps:

• input features → predicted output
• i.e., ŷ = f(x₁, …, xₙ)

The model involves:

• Parameters (learnable weights)

  • trainable values learned during training
  • example: linear regression uses coefficients (e.g., β₁, β₀)

• Hyperparameters (chosen before training; not directly learned)

  • determine model type/structure and training behavior
  • examples:
    • degree of polynomial (linear vs quadratic vs cubic) in regression
    • learning rate
    • neural network architecture choices (number of layers, nodes, etc.)

---

B) Training/test set workflow (and why split is necessary)

Start with a dataset of D points, where each point contains:

• a feature vector xᵢ
• a target value Yᵢ

Split the dataset into:

• Training set

  • used to fit/train the model parameters

• Test set

  • kept unseen until training is completed
  • used for independent verification of performance

Overfitting/underfitting guidance

• If you use too much data for training: risk of overfitting
• If you use too little data for training: risk of underfitting
• (Overfitting/underfitting will be covered later.)

---

C) Loss function and learning process (qualitative description)

During training, the model’s predictions should improve by reducing error. This is tracked using a loss function, which quantifies discrepancy between:

• predicted values ŷ
• actual values y

Goal: drive the loss lower as training proceeds.

A parity plot concept is referenced:

• points ideally lie near the line y = x (perfect predictions)

---

D) Cross-validation for generalization + hyperparameter selection

Cross-validation is presented as a method to test generalizability across different data splits.

Process described:

• Instead of one fixed train/test split:
  • create multiple realizations (different splits)
• Use:
  • a validation set (unseen during training, but used to evaluate/choose models)
• Compare model performance across splits to measure robustness.
• Use results to choose:
  • which model type works best (e.g., linear vs nonlinear, trees vs random forests vs deep learning)
  • and tune hyperparameters accordingly
• Then validate final choices using the test set.

---

What the next lecture will cover

The lecture ends by previewing:

• probability and statistics
• mean, standard deviation, variance
• how these connect to linear regression

---

Speakers / sources featured

• No specific individual speakers or external sources are named in the provided subtitles.