Feature Importance in Random Forests & XGBoost

Feature Importance as an XAI Tool — The Big Picture

In the XAI framework, feature importance sits at the intersection of transparency (what did the model learn?), interpretability (which inputs drove this decision?), and accountability (can a human verify those drivers are fair and legal?). It is the most widely deployed XAI primitive in production ML systems today.

The Black Box Problem — Why XAI Needs Feature Importance

XAI Scope — Global vs Local Explanations

Every feature importance method sits on a spectrum between global explanations (what the model learned overall) and local explanations (why this specific prediction was made). Understanding this distinction is crucial — regulators, data scientists, and end users each need a different scope.

MDI — XAI Through the Forest's Own Eyes

Permutation Importance — Model-Agnostic XAI

Random Forest Feature Importance — Full XAI Code

The following code produces a complete XAI report for a Random Forest model: global MDI importance, unbiased permutation importance on the validation set, and the OOB score as a proxy for generalisation. We use the Titanic dataset — a proxy for any high-stakes binary classification task (survive/don't survive → approved/denied).

import numpy as np
import pandas as pd
from sklearn.ensemble import RandomForestClassifier
from sklearn.inspection import permutation_importance
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelEncoder

# ── Data prep ─────────────────────────────────────────────────
df = pd.read_csv('titanic.csv')
df['Age'].fillna(df['Age'].median(), inplace=True)
df['Embarked'].fillna('S', inplace=True)
df['Sex']      = LabelEncoder().fit_transform(df['Sex'])
df['Embarked'] = LabelEncoder().fit_transform(df['Embarked'])

features = ['Pclass', 'Sex', 'Age', 'SibSp', 'Parch', 'Fare', 'Embarked']
X = df[features];  y = df['Survived']
X_train, X_val, y_train, y_val = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y)

# ── Train model ───────────────────────────────────────────────
rf = RandomForestClassifier(
    n_estimators=500, max_features='sqrt',
    min_samples_leaf=2, oob_score=True,
    n_jobs=-1, random_state=42
)
rf.fit(X_train, y_train)

# ══ XAI REPORT 1: Global MDI Importance ══════════════════════
mdi = pd.Series(rf.feature_importances_, index=features).sort_values(ascending=False)
print("══ XAI Report: MDI (Intrinsic Global Importance) ══")
for feat, imp in mdi.items():
    print(f"  {feat:12s}: {imp:.4f}  ({imp*100:.1f}% of model decisions)")

# ══ XAI REPORT 2: Permutation Importance (Model-Agnostic) ════
perm = permutation_importance(
    rf, X_val, y_val, n_repeats=20, random_state=42, n_jobs=-1)
perm_df = pd.DataFrame({
    'feature': features,
    'mean':    perm.importances_mean,
    'std':     perm.importances_std
}).sort_values('mean', ascending=False)

print("\n══ XAI Report: Permutation (Mean ± Uncertainty) ════")
for _, row in perm_df.iterrows():
    print(f"  {row['feature']:12s}: {row['mean']:.4f} ± {row['std']:.4f}")
print(f"\n  OOB Accuracy: {rf.oob_score_:.4f}")

# XAI Fairness check: are any protected attributes high-importance?
# In Titanic: Sex is #1 — expected (historically real). In a
# credit model: flag Gender/Race/Postcode if they appear.
protected = ['Sex']
for p in protected:
    imp_val = mdi[p]
    if imp_val > 0.05:
        print(f"\n⚠️  XAI FAIRNESS FLAG: '{p}' has {imp_val:.2%} importance.")
        print(   f"     Review whether use of this feature is legally permissible.")

XGBoost — Three XAI Lenses on the Same Model

import xgboost as xgb
from xgboost import XGBClassifier
import pandas as pd

# ── Train XGBoost ─────────────────────────────────────────────
xgb_model = XGBClassifier(
    n_estimators=300, learning_rate=0.05, max_depth=5,
    subsample=0.8, colsample_bytree=0.8,
    eval_metric='logloss', random_state=42
)
xgb_model.fit(X_train, y_train)

# ══ XAI REPORT: All Three XGBoost Lenses ═════════════════════
xai_results = {}
for imp_type in ['weight', 'gain', 'cover']:
    scores = xgb_model.get_booster().get_score(importance_type=imp_type)
    xai_results[imp_type] = scores
    ranked = sorted(scores.items(), key=lambda x: -x[1])
    print(f"\n══ XAI Lens: {imp_type.upper()} ════════════════════")
    for feat, val in ranked:
        print(f"  {feat:12s}: {val:.2f}")

# ── XAI Agreement Score: do all three lenses agree on top-3? ─
top3 = {}
for k, v in xai_results.items():
    top3[k] = set(sorted(v, key=lambda x: -v[x])[:3])

agreement = top3['weight'] & top3['gain'] & top3['cover']
print(f"\n══ XAI Consensus Top-3 (all lenses agree): {agreement}")

SHAP — The Gold Standard of XAI

SHAP (SHapley Additive exPlanations) is grounded in cooperative game theory. It is currently the most rigorous, fairest, and most widely accepted XAI method in both academia and industry. It is the only method that simultaneously satisfies the four mathematical axioms that any fair explanation must obey.

import shap
import numpy as np
import pandas as pd

# ── TreeExplainer — optimised for RF + XGBoost ───────────────
# Model-specific SHAP computation (O(TLD) not brute-force O(2^p))

explainer_rf  = shap.TreeExplainer(rf)
shap_vals_rf  = explainer_rf.shap_values(X_val)
# For binary classification: shap_vals_rf[1] = SHAP for class 1

explainer_xgb = shap.TreeExplainer(xgb_model)
shap_vals_xgb = explainer_xgb.shap_values(X_val)

# ══ XAI REPORT 1: Global SHAP Importance (mean |SHAP|) ════════
global_shap = pd.DataFrame({
    'feature':  features,
    'RF_SHAP':  np.abs(shap_vals_rf[1]).mean(axis=0),
    'XGB_SHAP': np.abs(shap_vals_xgb).mean(axis=0)
}).sort_values('RF_SHAP', ascending=False)

print("══ XAI Report: Global SHAP (mean |φⱼ| per feature) ══")
print(global_shap.to_string(index=False))

# ══ XAI REPORT 2: Local SHAP — Explain ONE individual ══════════
# This is the XAI output you hand to the affected person.
idx = 0  # passenger 0: Female, Pclass=1, Age=38
base_rate = explainer_xgb.expected_value
prediction = xgb_model.predict_proba(X_val.iloc[[idx]])[0][1]

print(f"\n══ XAI Local Report — Passenger {idx} ════════════════")
print(f"  Base rate (avg survival):  {base_rate:.3f}")
print(f"  Model prediction P(survive): {prediction:.3f}")
print(f"  Total SHAP shift:            {prediction - base_rate:+.3f}")
print(f"\n  Feature breakdown:")
for feat, sv in zip(features, shap_vals_xgb[idx]):
    arrow = "▲" if sv > 0 else "▼"
    effect = "raises" if sv > 0 else "lowers"
    print(f"  {arrow} {feat:12s}: {sv:+.4f}  ({effect} survival probability)")

Animated SHAP Beeswarm — The XAI Visualisation Standard

The SHAP beeswarm plot is the most information-dense single-chart XAI visualisation. It shows global importance (vertical axis), individual instance distributions (horizontal spread), feature value direction (colour), and density — all simultaneously.

XAI Method Comparison — The Full Matrix

Animated XAI Race — RF vs XGBoost Rankings

XAI Pitfalls — When Importance Scores Mislead

XAI in Regulated Industries — What Each Method Covers

XAI Production Pipeline — The Right Order

Golden Rules — XAI with Feature Importance