Column Transformer in Machine Learning | How to use ColumnTransformer in Sklearn

ColumnTransformer in scikit-learn (tutorial)

What the video covers

Problem motivation: different columns require different preprocessing (numerical imputation/scaling, ordinal encoding, nominal one‑hot encoding). Doing each column separately and then manually concatenating results is error‑prone and tedious.

• Introduces sklearn.compose.ColumnTransformer to apply column‑specific transformers in one coordinated object.
• Recommends combining ColumnTransformer with sklearn.pipeline.Pipeline (covered in the next video).

Dataset used in the demo

A synthetic DataFrame of ~100 patient records with these columns:

• Numerical: fever (with ~10% missing), age
• Ordinal categorical: cough severity (mild / moderate / strong)
• Nominal categorical: gender, city (4 cities)

Key scikit-learn classes and objects shown

• SimpleImputer — fill missing values (used on the fever column)
• StandardScaler — scale numeric features (applied to fever)
• OrdinalEncoder — encode ordered categories (used for cough; categories explicitly provided)
• OneHotEncoder (with drop=’first’) — encode nominal categories while avoiding multicollinearity (applied to gender and city)
• ColumnTransformer — combine multiple (name, transformer, column_list) tuples in one object
• Pipeline — mentioned as the next topic for chaining ColumnTransformer with an estimator

Practical steps demonstrated

1. Inspect the data and identify column types (numerical, ordinal, nominal).
2. Create individual transformers:
   • Example: imputer + scaler pipeline for fever
   • OrdinalEncoder for cough (with explicit category order)
   • OneHotEncoder(drop=’first’) for gender and city
3. Manually fit_transform each transformer on its column(s) to illustrate how tedious and error‑prone that is.
4. Build a ColumnTransformer by passing a list of tuples: (name, transformer_object, [column_names]).
   • Example tuple names used: tm1, tm2, tm3
   • Showed use of remainder=’passthrough’ to keep untransformed columns (alternative: remainder=’drop’)
5. Call column_transformer.fit_transform(train_df) and column_transformer.transform(test_df) to get the processed feature matrix.
6. Examine resulting shapes and feature order; observe effects of options like drop=’first’ in OneHotEncoder.
7. Recommend combining ColumnTransformer with Pipeline for streamlined model training (promised in the next video).

Benefits emphasized

• Centralizes and documents per‑column preprocessing.
• Keeps transformed columns aligned and prevents manual concatenation mistakes.
• Scales well when many columns require different preprocessing rules.
• Simple to reuse the same transformation on train/test splits.

Actionable advice

• Practice applying ColumnTransformer to your own datasets.
• For ordinal variables, explicitly pass the category order to OrdinalEncoder.
• Use remainder=’passthrough’ to keep other columns unchanged, or remainder=’drop’ to remove them.
• Use OneHotEncoder(drop=’first’) to reduce one dummy column per categorical feature and avoid multicollinearity.

Main speaker / source

• Video presenter: a YouTube instructor (unnamed in subtitles) — tutorial/demo style walkthrough using scikit‑learn.