Introduction to Federated Learning: Privacy-First Machine Learning at Scale

What is Federated Learning?

Federated Learning (FL) is a machine learning paradigm where a model is trained across many decentralised devices or servers holding local data samples, without exchanging the raw data itself. The term was coined by Google researchers McMahan et al. in 2017, originally to train the Gboard keyboard prediction model on Android phones.

Centralized vs. Federated Training

To appreciate federated learning, you must first deeply understand what it replaces and why the traditional approach breaks down at scale and in sensitive domains.

Why Federated Learning Matters

Federated Learning isn't a minor engineering tweak — it's a fundamental rethinking of who owns data and where intelligence should live. Two forces make it indispensable: privacy and scale.

🔒 The Privacy Argument

🏠 The Scale Argument

The second reason FL matters is raw scale. Modern ML needs enormous, diverse datasets. But the most valuable data in the world exists on billions of edge devices — phones, wearables, factory sensors, medical equipment — that will never centralise their data.

Real-World Use Cases

Federated Learning has moved well beyond the research lab. Here are the four most impactful domains where it is actively deployed today.

Key Players: Server, Clients, and Coordinator

Every federated system has three categories of actor. Understanding each one's role, responsibilities, and constraints is essential before you write a single line of FL code.

The FedAvg Algorithm — The Heart of FL

The Federated Averaging (FedAvg) algorithm, introduced by McMahan et al. (2017), is the foundation of almost every federated system deployed today. Understanding it precisely is essential before moving to more advanced FL methods.

💻 Minimal FedAvg Implementation in Python

import numpy as np
import copy
from collections import OrderedDict
import torch
import torch.nn as nn

# ── Simple model definition ──────────────────────────────
class SimpleNet(nn.Module):
    def __init__(self, input_dim=20, hidden=64, num_classes=2):
        super().__init__()
        self.net = nn.Sequential(
            nn.Linear(input_dim, hidden),
            nn.ReLU(),
            nn.Linear(hidden, num_classes)
        )
    def forward(self, x):
        return self.net(x)

# ── Local training on a single client ────────────────────
def local_train(model, data_loader, lr=0.01, epochs=3):
    model = copy.deepcopy(model)   # each client gets its own copy
    optimizer = torch.optim.SGD(model.parameters(), lr=lr)
    criterion = nn.CrossEntropyLoss()

    model.train()
    for _ in range(epochs):
        for X, y in data_loader:
            optimizer.zero_grad()
            loss = criterion(model(X), y)
            loss.backward()
            optimizer.step()
    return model.state_dict()        # return updated weights

# ── FedAvg aggregation ────────────────────────────────────
def fedavg(global_weights, client_weights, client_sizes):
    """Weighted average of client model weights by dataset size."""
    total = sum(client_sizes)
    averaged = copy.deepcopy(global_weights)

    # Initialise all tensors to zero
    for key in averaged:
        averaged[key] = torch.zeros_like(averaged[key], dtype=torch.float)

    # Weighted sum across all clients
    for w, n in zip(client_weights, client_sizes):
        for key in averaged:
            averaged[key] += (n / total) * w[key].float()

    return averaged

# ── Federated training loop ───────────────────────────────
def federated_train(global_model, client_data_loaders,
                     rounds=10, fraction=0.5):
    """
    global_model      : initialised PyTorch model
    client_data_loaders: list of DataLoaders, one per client
    rounds            : number of FL communication rounds
    fraction          : fraction of clients selected per round
    """
    num_clients = len(client_data_loaders)

    for r in range(rounds):
        # Step 1: select a subset of clients (coordinator logic)
        selected_idx = np.random.choice(
            num_clients,
            size=max(1, int(fraction * num_clients)),
            replace=False
        )

        client_weights = []
        client_sizes   = []

        # Step 2: each selected client trains locally
        for idx in selected_idx:
            local_w = local_train(global_model, client_data_loaders[idx])
            client_weights.append(local_w)
            client_sizes.append(len(client_data_loaders[idx].dataset))

        # Step 3: server aggregates with FedAvg
        new_global_weights = fedavg(
            global_model.state_dict(), client_weights, client_sizes
        )
        global_model.load_state_dict(new_global_weights)

        print(f"Round {r+1}/{rounds} complete — {len(selected_idx)} clients trained")

    return global_model

Three Types of Federated Learning

Not all federated learning systems look the same. The differences in data distribution lead to three distinct FL paradigms. Knowing which type you need determines your entire architecture.

Summary & What You Now Know

You have covered the complete conceptual foundation of Federated Learning. Here is what to carry forward: