Transfer Learning in NLP

The Story That Explains Transfer Learning

Before transfer learning took over NLP around 2018, every new task required training a model from scratch. You needed millions of labelled examples and weeks of GPU time for each project. Most organisations simply could not afford it. Transfer learning changed all of that in one decade.

Why NLP Needed Transfer Learning So Badly

Language is brutally hard to learn from nothing. Consider what a model must know just to understand one sentence: "The bank refused the loan because it was insolvent." It needs to know that bank here means a financial institution (not a riverbank), that it refers to the bank (not the loan), and that insolvent means unable to pay debts. That one sentence requires finance knowledge, entity resolution, and vocabulary depth.

Building all of that from 10,000 labelled sentiment examples is impossible. But a model pre-trained on 800 GB of text has already internalised most of it. Your job becomes dramatically simpler.

The Evolution: Word2Vec → ELMo → BERT → GPT

How Pretraining Works — The Two Objectives

All transfer learning begins with pretraining — training a model on a massive self-supervised task so it learns language structure without needing human labels. Two pretraining objectives dominate the field.

The Fine-Tuning Paradigm — Adapting a Pretrained Model

Fine-tuning involves taking a pretrained model and continuing training it on your labelled task-specific dataset — but with a much lower learning rate. We don't want to destroy the knowledge gained during pretraining; we want to gently steer the model toward the new task.

BERT Architecture — A Visual Tour

Understanding BERT's architecture tells you why it transfers so well. Every layer learns increasingly abstract language features — from token-level spelling at the bottom to semantic reasoning at the top.

Three Strategies for Fine-Tuning

Not all transfer learning looks the same. Depending on your dataset size and how different your task is from the original pretraining, you'll choose one of three strategies.

LoRA — Low-Rank Adaptation of Large Language Models

The mathematical insight behind LoRA: weight updates during fine-tuning tend to be low-rank. Instead of updating the full weight matrix W (which could be 4096 × 4096 = 16.7 million numbers), you decompose the update as two small matrices: ΔW = A × B, where A is 4096×r and B is r×4096, with r typically 4–64.

What Each Transformer Layer Actually Learns

Probing experiments — training small classifiers on each layer's representations — reveal a consistent pattern across BERT and its descendants. This is why transfer learning works so powerfully.

Code — Full Fine-Tuning with Hugging Face Transformers

The Hugging Face transformers library is the standard toolkit for transfer learning in NLP. Below is a complete, production-ready sentiment classification pipeline using bert-base-uncased on the IMDB movie review dataset.

# ── Install: pip install transformers datasets accelerate ─────────────

import torch
from datasets import load_dataset
from transformers import (
    AutoTokenizer,
    AutoModelForSequenceClassification,
    TrainingArguments,
    Trainer,
    DataCollatorWithPadding,
)
import numpy as np
from sklearn.metrics import accuracy_score, f1_score

# ── 1. Load dataset ────────────────────────────────────────────────────
dataset = load_dataset("imdb")
# Shrink to 2000 train / 500 test for fast demo
dataset["train"] = dataset["train"].select(range(2000))
dataset["test"]  = dataset["test"].select(range(500))

# ── 2. Load tokeniser ──────────────────────────────────────────────────
MODEL_NAME = "bert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)

def tokenize(batch):
    return tokenizer(
        batch["text"],
        truncation=True,
        max_length=256,       # IMDB reviews are long; truncate to save memory
        padding=False,        # dynamic padding via DataCollator
    )

tokenized = dataset.map(tokenize, batched=True)
data_collator = DataCollatorWithPadding(tokenizer=tokenizer)

# ── 3. Load pretrained model and add classification head ───────────────
model = AutoModelForSequenceClassification.from_pretrained(
    MODEL_NAME,
    num_labels=2,            # pos / neg
    id2label={0: "NEGATIVE", 1: "POSITIVE"},
    label2id={"NEGATIVE": 0, "POSITIVE": 1},
)

# ── 4. Metrics function ─────────────────────────────────────────────────
def compute_metrics(eval_pred):
    logits, labels = eval_pred
    preds = np.argmax(logits, axis=-1)
    return {
        "accuracy": accuracy_score(labels, preds),
        "f1":       f1_score(labels, preds, average="binary"),
    }

# ── 5. Training arguments ───────────────────────────────────────────────
args = TrainingArguments(
    output_dir="./bert-imdb",
    evaluation_strategy="epoch",
    save_strategy="epoch",
    learning_rate=2e-5,      # critical: much smaller than standard SGD
    per_device_train_batch_size=16,
    per_device_eval_batch_size=32,
    num_train_epochs=3,
    weight_decay=0.01,
    load_best_model_at_end=True,
    metric_for_best_model="f1",
    warmup_ratio=0.1,        # warm up LR for first 10% of steps
    fp16=torch.cuda.is_available(),  # mixed precision if GPU available
    logging_steps=50,
    seed=42,
)

# ── 6. Trainer ──────────────────────────────────────────────────────────
trainer = Trainer(
    model=model,
    args=args,
    train_dataset=tokenized["train"],
    eval_dataset=tokenized["test"],
    tokenizer=tokenizer,
    data_collator=data_collator,
    compute_metrics=compute_metrics,
)

trainer.train()
results = trainer.evaluate()
print(f"Final Test Accuracy: {results['eval_accuracy']:.4f}")
print(f"Final Test F1:       {results['eval_f1']:.4f}")

Code — LoRA Fine-Tuning with PEFT for Large Models

When your model is 7B+ parameters, full fine-tuning is often impractical. LoRA via the Hugging Face peft library makes it trivial. The example below fine-tunes a small GPT-2 with LoRA for demonstration — the same code scales directly to LLaMA-3-8B or Mistral-7B.

# ── Install: pip install transformers peft datasets ───────────────────

from peft import (
    LoraConfig,
    TaskType,
    get_peft_model,
    PeftModel,
)
from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch

# ── Load base model ────────────────────────────────────────────────────
BASE_MODEL = "gpt2"         # swap with "meta-llama/Meta-Llama-3-8B" etc.
tokenizer  = AutoTokenizer.from_pretrained(BASE_MODEL)
tokenizer.pad_token = tokenizer.eos_token

model = AutoModelForSequenceClassification.from_pretrained(
    BASE_MODEL,
    num_labels=2,
    torch_dtype=torch.float16,   # half precision to fit in GPU memory
)
model.config.pad_token_id = tokenizer.eos_token_id

# ── Define LoRA config ─────────────────────────────────────────────────
lora_config = LoraConfig(
    task_type=TaskType.SEQ_CLS,
    r=16,                 # rank — higher = more capacity, more params
    lora_alpha=32,        # scaling factor; alpha = 2*r is a good default
    lora_dropout=0.1,    # regularisation for LoRA layers
    target_modules=["c_attn", "c_proj"],  # GPT-2 attention projection names
    bias="none",
)

# ── Wrap model with LoRA ───────────────────────────────────────────────
peft_model = get_peft_model(model, lora_config)
peft_model.print_trainable_parameters()

# ── Inspect: how many parameters are trainable? ────────────────────────
total   = sum(p.numel() for p in peft_model.parameters())
trained = sum(p.numel() for p in peft_model.parameters() if p.requires_grad)
print(f"Trainable params: {trained:,} / {total:,}  ({100*trained/total:.2f}%)")

# ── Save and reload LoRA adapter (tiny file!) ──────────────────────────
peft_model.save_pretrained("./lora-adapter")
# Saved file is typically 5–50 MB even for 7B models
# Reload: merged = PeftModel.from_pretrained(base_model, "./lora-adapter")

Common NLP Tasks — Which Pretrained Model to Choose

Different tasks have different ideal model architectures. Encoder-only models excel at understanding; decoder-only at generation; encoder-decoder at transformation tasks.

Catastrophic Forgetting — The Hidden Danger

Multilingual Transfer Learning — One Model, 100 Languages

One of the most remarkable feats of modern NLP is zero-shot cross-lingual transfer. Fine-tune a multilingual model on English labelled data, and it immediately works in Hindi, Swahili, or Arabic — without a single labelled example in those languages.

Code — Named Entity Recognition (NER) Fine-Tuning

Token classification is one of the most common industrial NLP tasks — extracting names, dates, organisations, locations from text. Here's a concise fine-tuning example.

from transformers import (
    AutoTokenizer,
    AutoModelForTokenClassification,
    TrainingArguments,
    Trainer,
    DataCollatorForTokenClassification,
)
from datasets import load_dataset
import numpy as np
from seqeval.metrics import classification_report

# ── Load CoNLL-2003 NER dataset ────────────────────────────────────────
dataset   = load_dataset("conll2003")
label_list = dataset["train"].features["ner_tags"].feature.names
print(f"NER Labels: {label_list}")
# ['O', 'B-PER', 'I-PER', 'B-ORG', 'I-ORG', 'B-LOC', 'I-LOC', 'B-MISC', 'I-MISC']

MODEL_NAME = "roberta-base"
tokenizer  = AutoTokenizer.from_pretrained(MODEL_NAME, add_prefix_space=True)

def tokenize_and_align_labels(examples):
    """Tokenize and re-align NER labels with subword tokens."""
    tokenized_inputs = tokenizer(
        examples["tokens"],
        truncation=True,
        is_split_into_words=True,  # input is already word-tokenised
    )
    all_labels = []
    for i, labels in enumerate(examples["ner_tags"]):
        word_ids      = tokenized_inputs.word_ids(batch_index=i)
        aligned       = []
        previous_word = None
        for word_id in word_ids:
            if word_id is None:
                aligned.append(-100)       # ignore [CLS], [SEP], padding
            elif word_id != previous_word:
                aligned.append(labels[word_id])
            else:
                aligned.append(-100)       # ignore continuation subwords
            previous_word = word_id
        all_labels.append(aligned)
    tokenized_inputs["labels"] = all_labels
    return tokenized_inputs

tokenized = dataset.map(tokenize_and_align_labels, batched=True)
collator  = DataCollatorForTokenClassification(tokenizer=tokenizer)

# ── Model ──────────────────────────────────────────────────────────────
model = AutoModelForTokenClassification.from_pretrained(
    MODEL_NAME,
    num_labels=len(label_list),
    id2label={i: l for i, l in enumerate(label_list)},
    label2id={l: i for i, l in enumerate(label_list)},
)

args = TrainingArguments(
    output_dir="./roberta-ner",
    evaluation_strategy="epoch",
    learning_rate=3e-5,
    per_device_train_batch_size=32,
    num_train_epochs=4,
    weight_decay=0.01,
)

trainer = Trainer(
    model=model, args=args,
    train_dataset=tokenized["train"],
    eval_dataset=tokenized["validation"],
    data_collator=collator,
    tokenizer=tokenizer,
)
trainer.train()

The BERT Family — Which Variant for Your Constraints?

Domain-Adaptive Pretraining — Closing the Domain Gap

BERT was pretrained on Wikipedia and Books. If your task involves biomedical literature, legal contracts, or financial filings, there is a domain gap: the model has never seen the vocabulary, abbreviations, or writing style of your domain.

Transfer Learning vs Training from Scratch

Golden Rules of Transfer Learning in NLP

Quick Reference — Transfer Learning Decision Cheatsheet