EDA using Univariate Analysis | Day 20 | 100 Days of Machine Learning

Main ideas & lessons (Exploratory Data Analysis with univariate analysis)

• The purpose of Exploratory Data Analysis (EDA) is to understand the data before modeling.
• EDA can be approached in:
  • Univariate analysis: analyzing one variable/column at a time.
  • Multivariate analysis: analyzing multiple variables/columns together (noted as the next topic after univariate).

What “variable” means in a dataset

• A dataset (data frame) has multiple columns, and each column is a variable.
  • Single column → univariate
  • Two columns → discussed as multivariate
  • More than two columns → multivariate

Data types to classify columns (before plotting)

• Numerical data (quantitative, continuous/ordered) Examples: height, weight, phone price, battery life/capacity.

• Categorical data (discrete labels; can be nominal or ordinal) Examples: country, nationality, state, gender, college, engineering branch.

Methodology / workflow for univariate EDA

Step-by-step workflow

1. Inspect dataset columns
   • Identify which columns are categorical vs numerical.
2. Run univariate analysis column-by-column
   • The focus is on analyzing each column independently, starting with categorical columns.

---

A) Univariate EDA for categorical columns

Recommended plots/approach

• Use value counts to get the frequency of each category.
• Use an account/frequency bar plot (bar chart-style counts).
• Optionally use a pie chart to show category proportions (with percentages displayed).

What to conclude from categorical plots

• The total number of records and the breakdown by category.
• How category distribution relates to the target concept (e.g., using relevant columns to infer patterns like survival vs not survival).

Example demonstrations using the Titanic dataset

• Survival (target)
  • Count how many passengers survived vs did not survive.
• Pclass (passenger class)
  • Determine which class is most/least represented.
• Sex (gender)
  • Determine gender distribution (and corresponding percentages).
• Embarked (boarding location)
  • Determine how many passengers boarded from each location.
• Fare (ticket cost)
  • Mentioned as something you can examine (e.g., via grouping/summary), though the main detailed focus is on frequency/proportion-based plots.

---

B) Univariate EDA for numerical columns

Main idea

• Numerical data is continuous, so you want to understand its distribution.

Recommended plots

• Histogram
  • Use ranges/bins between the minimum and maximum values.
  • Shows how many values fall into each numeric interval (reveals distribution shape).
  • Bin/range settings can be adjusted to change granularity.
• Kernel Density Estimation (KDE) / density plot
  • Produces a smooth estimate of the probability density.
  • Intuition is explained using:
    • PDF (probability density function) and probability intuition across x-values.
• Box plot
  • Provides a five-number summary and helps detect outliers.
  • Key concepts for interpreting box plots:
    • Median
    • Percentiles (Q1, Q3 interpretation)
    • Minimum/maximum and their relation to quartiles
    • Interquartile range (IQR)
    • Outlier detection using computed fences (values outside the expected range)

What to conclude from numerical plots

• For histogram/KDE:
  • Where values concentrate (middle vs extremes)
  • Whether the distribution looks symmetric or skewed
  • Outliers and density concentration
• For box plot:
  • Median location
  • Spread (IQR and whiskers)
  • Presence of outliers / abnormal values

---

Distribution shape checks (quality/interpretation hints)

• The video discusses symmetry and skewness:
  • If data is symmetric, it resembles a normal distribution.
  • If skewed, values may concentrate toward one side (lower or higher values).
• Skewness can be tested numerically/statistically:
  • Skewness > 0 → skewed to the right
  • Skewness < 0 → skewed to the left

---

Dataset used in the video

• Titanic dataset
  • Each row represents a passenger.
  • Target concept: Survival
    • 0 = did not survive, 1 = survived
  • Columns mentioned (some partially/garbled in the subtitles) include:
    • passenger ID, pclass, sibsp (spouse/partner count), parch (parents/children count), fare, age, sex, embarked, etc.

Speakers / sources featured

• Primary speaker (YouTube creator): an unnamed instructor/host (explains concepts and demonstrates code/plots).
• Source dataset: Titanic dataset.
• Libraries/tools mentioned
  • pandas (e.g., value_counts, dataframe column operations)
  • matplotlib / pyplot (implied for plotting)
  • seaborn (explicitly referenced for histogram-style and KDE-style plots)