One-Hot Encoding

The Story That Explains One-Hot Encoding

One-Hot Encoding is the process of converting a categorical variable with n unique categories into n binary columns (or n−1 if using the dummy variable trap prevention). Each row gets a 1 in exactly one of those new columns and 0 everywhere else.

What Is a Categorical Variable?

Before encoding, you need to know what you are encoding. Categorical variables come in two flavours, and confusing them is a costly mistake.

The Mechanics — How One-Hot Encoding Works

Let us walk through the transformation step by step using a concrete dataset. We have a column called Colour with three unique values: Red, Blue, Green.

Step 1 — The Original Column (Before)

Notice: each row has exactly one 1 and all other columns are 0. The original column disappears. Three categories → three new binary columns.

The Dummy Variable Trap ⚠️

When both Colour_Red and Colour_Blue are 0, the model correctly infers Green without needing an explicit column. The dropped category becomes the reference category.

Visual Diagram — The Encoding Pipeline

Python Implementation — pd.get_dummies()

The simplest and most commonly used method in Python is pd.get_dummies() from the Pandas library. It works directly on a DataFrame and handles everything for you.

import pandas as pd

# ── Sample dataset ─────────────────────────────────────────
data = {
    'Name':   ['Alice', 'Bob', 'Carol', 'Dave', 'Eve'],
    'Colour': ['Red', 'Blue', 'Green', 'Red', 'Green'],
    'Score':  [85, 92, 78, 95, 88]
}
df = pd.DataFrame(data)
print("Original DataFrame:")
print(df)

# ── Basic One-Hot Encoding ──────────────────────────────────
df_encoded = pd.get_dummies(df, columns=['Colour'])
print("\nAfter get_dummies():")
print(df_encoded)

# ── Drop first to avoid dummy variable trap ─────────────────
df_nodrop = pd.get_dummies(df, columns=['Colour'], drop_first=True)
print("\nWith drop_first=True (avoids trap):")
print(df_nodrop)

# ── Check data types (convert bool to int if needed) ────────
df_int = pd.get_dummies(df, columns=['Colour'], dtype=int)
print("\nWith dtype=int (0/1 instead of True/False):")
print(df_int.dtypes)

Python Implementation — sklearn OneHotEncoder

For production pipelines and ML workflows, sklearn.preprocessing.OneHotEncoder is the recommended approach. It integrates cleanly into Pipeline and ColumnTransformer, handles unseen categories gracefully, and supports .fit() / .transform() split — essential for preventing data leakage.

import pandas as pd
import numpy as np
from sklearn.preprocessing import OneHotEncoder
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LogisticRegression

# ── Dataset ─────────────────────────────────────────────────
df = pd.DataFrame({
    'Colour':  ['Red', 'Blue', 'Green', 'Red', 'Blue'],
    'Size':    ['S', 'L', 'M', 'XL', 'S'],
    'Price':   [10.5, 22.0, 15.0, 30.0, 9.5],
    'Sold':    [1, 0, 1, 1, 0]
})

X = df[['Colour', 'Size', 'Price']]
y = df['Sold']

# ── Define which columns get encoded ────────────────────────
categorical_cols = ['Colour', 'Size']
numeric_cols     = ['Price']

# ── ColumnTransformer: apply OHE to categoricals only ───────
preprocessor = ColumnTransformer(transformers=[
    ('ohe', OneHotEncoder(
        drop='first',            # avoid dummy trap
        handle_unknown='ignore', # unseen categories → all zeros
        sparse_output=False      # return dense array
    ), categorical_cols),
    ('num', 'passthrough', numeric_cols)
])

# ── Full ML Pipeline ─────────────────────────────────────────
pipe = Pipeline(steps=[
    ('preprocessor', preprocessor),
    ('classifier', LogisticRegression(max_iter=200))
])

pipe.fit(X, y)

# ── See what feature names were created ──────────────────────
ohe_features = pipe.named_steps['preprocessor'] \
                   .named_transformers_['ohe'] \
                   .get_feature_names_out(categorical_cols)
print("OHE Features:", ohe_features)
print("All Features: ", list(ohe_features) + numeric_cols)

# ── Predict on new (unseen) data ─────────────────────────────
new_data = pd.DataFrame({
    'Colour': ['Purple'],  # unseen category — handle_unknown='ignore' saves us
    'Size':   ['M'],
    'Price':  [18.0]
})
print("\nPrediction for unseen colour 'Purple':", pipe.predict(new_data))

Real-World Example — Titanic Survival Prediction

import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import OneHotEncoder
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score, classification_report

# ── Load Titanic (from seaborn for convenience) ──────────────
import seaborn as sns
df = sns.load_dataset('titanic')

# ── Select features — mix of categorical and numeric ────────
features = ['pclass', 'sex', 'age', 'sibsp', 'parch', 'fare', 'embarked']
target    = 'survived'

df = df[features + [target]].dropna()
X = df[features]
y = df[target]

# ── Identify column types ────────────────────────────────────
cat_cols = ['sex', 'embarked']   # nominal → OneHotEncoder
num_cols = ['pclass', 'age', 'sibsp', 'parch', 'fare']

# ── Train / test split ───────────────────────────────────────
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)

# ── Build preprocessor ──────────────────────────────────────
preprocessor = ColumnTransformer([
    ('ohe', OneHotEncoder(drop='first', handle_unknown='ignore'), cat_cols),
    ('pass', 'passthrough', num_cols)
])

# ── Full pipeline ────────────────────────────────────────────
model = Pipeline([
    ('prep', preprocessor),
    ('clf', RandomForestClassifier(n_estimators=200, random_state=42, n_jobs=-1))
])

model.fit(X_train, y_train)
y_pred = model.predict(X_test)

print(f"Accuracy: {accuracy_score(y_test, y_pred):.4f}")
print("\nClassification Report:")
print(classification_report(y_test, y_pred))

# ── Show what OHE created ────────────────────────────────────
ohe_names = model['prep'].named_transformers_['ohe'].get_feature_names_out(cat_cols)
all_names  = list(ohe_names) + num_cols
print("\nFinal feature names after OHE:")
print(all_names)

Encoding Methods Comparison — When to Use What

One-Hot Encoding is not the only encoding method. Understanding when not to use it is as important as knowing when to use it.

The Curse of Dimensionality

import pandas as pd

# ── Diagnose your categorical column before encoding ────────
def diagnose_categorical(series):
    n_unique    = series.nunique()
    value_counts = series.value_counts()
    rare_pct    = (value_counts < 30).sum() / n_unique * 100

    print(f"Column:       {series.name}")
    print(f"Unique vals:  {n_unique}")
    print(f"Rare cats:    {rare_pct:.1f}% (freq < 30)")

    if n_unique <= 15:
        print("Recommendation: ✅ One-Hot Encoding is safe")
    elif n_unique <= 50:
        print("Recommendation: ⚠️  Consider Binary or Target Encoding")
    else:
        print("Recommendation: ❌ Do NOT use One-Hot — use Target/Hashing")

# ── Examples ──────────────────────────────────────────────────
df = pd.DataFrame({
    'Colour':  ['Red', 'Blue', 'Green'] * 100,
    'City':    [f"City_{i}" for i in range(300)]
})
diagnose_categorical(df['Colour'])
print()
diagnose_categorical(df['City'])

Handling Unseen Categories in Production

One of the most common production bugs: the encoder was fitted on training data with colour values Red, Blue, Green — then in production a user submits "Purple." What happens?

from sklearn.preprocessing import OneHotEncoder
import numpy as np

# ── Training data ────────────────────────────────────────────
X_train = np.array([['Red'], ['Blue'], ['Green'], ['Red']])

# ── Test data includes 'Purple' — not seen during training ──
X_test  = np.array([['Purple'], ['Blue'], ['Yellow']])

# ── handle_unknown='ignore' → all zeros for unseen ──────────
ohe = OneHotEncoder(handle_unknown='ignore', sparse_output=False)
ohe.fit(X_train)

result = ohe.transform(X_test)
print("Encoded (ignore):")
print(pd.DataFrame(result, columns=ohe.get_feature_names_out()))

# ── handle_unknown='infrequent_if_exist' ────────────────────
ohe2 = OneHotEncoder(
    handle_unknown='infrequent_if_exist',
    min_frequency=2,   # categories seen fewer than 2 times → infrequent bucket
    sparse_output=False
)
ohe2.fit(X_train)
result2 = ohe2.transform(X_test)
print("\nEncoded (infrequent_if_exist):")
print(pd.DataFrame(result2, columns=ohe2.get_feature_names_out()))

Sparse Matrices — Memory Efficiency

When you One-Hot encode a column with many categories, the resulting matrix is extremely sparse — most values are 0. Storing all those zeros wastes memory. Sparse matrices store only the non-zero positions.

from sklearn.preprocessing import OneHotEncoder
import numpy as np

# ── sparse_output=True saves memory for large datasets ──────
ohe_sparse = OneHotEncoder(sparse_output=True)   # default in sklearn
ohe_dense  = OneHotEncoder(sparse_output=False)  # returns numpy array

X = np.array([['Red'], ['Blue'], ['Green']] * 1000)

sparse_result = ohe_sparse.fit_transform(X)
dense_result  = ohe_dense.fit_transform(X)

import sys
print(f"Sparse matrix memory: {sparse_result.data.nbytes:,} bytes")
print(f"Dense  array  memory: {dense_result.nbytes:,} bytes")
print(f"Memory saving: {dense_result.nbytes / sparse_result.data.nbytes:.1f}x")

Common Mistakes and How to Avoid Them

Which Algorithms Require One-Hot Encoding?

Golden Rules for One-Hot Encoding