Estimation Theory

The Archaeologist, The Doctor & The Opinion Poll

In 1991, hikers in the Alps stumbled upon a 5,300-year-old frozen corpse — now famous as Ötzi the Iceman. Scientists could not travel back in time to watch him die. But using carbon-14 decay rates from bone samples, they estimated his age of death at approximately 45 years old. Not a guess. Not a certainty. A precise, evidence-based estimate with a measurable margin of error.

A cardiologist measures a patient's blood pressure three times and reports: "Your average systolic pressure is 138 mmHg — we should investigate." She did not measure every heartbeat in the patient's lifetime. She took a sample and made an inference about the patient's true underlying pressure.

Estimation theory is the formal statistical framework that tells us how to make those estimates, how precise they are, and how confident we should be in them. It is the engine behind clinical trials, quality control, A/B testing, credit risk models, and virtually every data-driven decision.

The Two Branches of Estimation

When you collect a sample and want to say something about the population, you have two tools: give a single best-guess number, or give a range of plausible values. These are the two branches of estimation — and you almost always need both.

Point Estimation — The Best Single Guess

A point estimator is a formula (a statistic) that uses sample data to produce a single number as the best estimate of an unknown population parameter. The most common estimators are:

Properties of a Good Estimator

Interval Estimation & Confidence Intervals ⭐

A point estimate is useful but incomplete — it tells you the best single guess but says nothing about how uncertain that guess is. A sample mean of ₹42,000 from 10 workers feels much less reliable than ₹42,000 from 10,000 workers. Interval estimation captures that uncertainty by constructing a range of plausible values for the parameter.

Margin of Error — The ± Number Everyone Sees

The Margin of Error (MoE) is the half-width of a confidence interval. It is the ±3% you see in election polls, the ±0.5°C in weather forecasts, the ±2 kg in clinical trials. It tells you how far your point estimate might stray from the true population value in either direction.

Step-by-Step: Building a Confidence Interval

A quality engineer samples 36 LED bulbs from a production line and measures their lifespan (hours). She wants to estimate the true average lifespan of all bulbs produced with a 95% confidence interval.

Python Implementation

Point Estimation

import numpy as np
from scipy import stats

np.random.seed(42)

# Sample: lifespan of 36 LED bulbs (hours)
sample = np.random.normal(loc=1480, scale=120, size=36)

# Point estimates
x_bar = np.mean(sample)             # estimates μ
s     = np.std(sample, ddof=1)      # estimates σ (unbiased, ddof=1)
n     = len(sample)

print(f"Point estimate of μ (sample mean x̄): {x_bar:.2f} hrs")
print(f"Point estimate of σ (sample std  s):  {s:.2f} hrs")
print(f"Sample size n: {n}")
# Output (approx):
# Point estimate of μ (sample mean x̄): 1476.84 hrs
# Point estimate of σ (sample std  s):  118.43 hrs

Confidence Interval — Using SciPy t-Distribution

from scipy import stats
import numpy as np

np.random.seed(42)
sample = np.random.normal(loc=1480, scale=120, size=36)

x_bar = np.mean(sample)
s     = np.std(sample, ddof=1)
n     = len(sample)
se    = s / np.sqrt(n)

# 95% CI using t-distribution (σ unknown)
ci_95 = stats.t.interval(confidence=0.95, df=n-1, loc=x_bar, scale=se)

# 99% CI
ci_99 = stats.t.interval(confidence=0.99, df=n-1, loc=x_bar, scale=se)

# Critical t* values
t_95 = stats.t.ppf(0.975, df=n-1)
t_99 = stats.t.ppf(0.995, df=n-1)

print(f"Standard Error (SE): {se:.2f} hrs")
print(f"t* for 95% CI (df=35): {t_95:.4f}")
print(f"Margin of Error 95%: ±{t_95 * se:.2f} hrs")
print(f"95% CI: ({ci_95[0]:.2f}, {ci_95[1]:.2f}) hrs")
print(f"99% CI: ({ci_99[0]:.2f}, {ci_99[1]:.2f}) hrs")

Confidence Interval for a Proportion

import numpy as np
from scipy import stats

# Scenario: 420 out of 1,000 voters prefer Party A
n       = 1000
x       = 420          # successes
p_hat   = x / n        # 0.42

se_prop = np.sqrt(p_hat * (1 - p_hat) / n)

# 95% CI for proportion (z-interval — n is large)
z_star = stats.norm.ppf(0.975)           # 1.95996...
moe    = z_star * se_prop
ci_low = p_hat - moe
ci_hi  = p_hat + moe

print(f"Sample proportion p̂:    {p_hat:.4f} ({p_hat*100:.1f}%)")
print(f"Standard Error:          {se_prop:.4f}")
print(f"Critical z*:             {z_star:.4f}")
print(f"Margin of Error:         ±{moe:.4f} (±{moe*100:.2f}%)")
print(f"95% CI: ({ci_low*100:.2f}%, {ci_hi*100:.2f}%)")

# Output:
# Sample proportion p̂:    0.4200 (42.0%)
# Standard Error:          0.0156
# Critical z*:             1.9600
# Margin of Error:         ±0.0306 (±3.06%)
# 95% CI: (38.94%, 45.06%)

Required Sample Size Calculation

import numpy as np
from scipy import stats

# How large a sample do we need for a desired Margin of Error?

z_star   = stats.norm.ppf(0.975)    # 1.96 for 95% CI
sigma    = 120                       # estimated std dev (from pilot study)
desired_moe = 25                     # we want MoE ≤ 25 hours

# n = (z* × σ / MoE)²
n_required = (z_star * sigma / desired_moe) ** 2

print(f"Required sample size: n ≥ {np.ceil(n_required):.0f}")
# Output: Required sample size: n ≥ 89

# For proportion — worst case p̂ = 0.5
desired_moe_prop = 0.03              # ±3%
n_prop = (z_star * 0.5 / desired_moe_prop) ** 2
print(f"Required n for ±3% prop CI: n ≥ {np.ceil(n_prop):.0f}")
# Output: Required n for ±3% prop CI: n ≥ 1068

Estimation Methods — Comparison Table

Golden Rules