tf.summary and TensorBoard Logging

Section 01

The Story — The Aeroplane's Flight Recorder

📖 Real World Analogy

Record Everything, Look Later

Every aeroplane carries a flight recorder. It quietly writes down speed, height and engine data every second. After the flight, engineers open it and see exactly what happened, when, and why.

Training a model for hours is a long flight. tf.summary is your flight recorder: it writes numbers, histograms, images and text to small log files. TensorBoard is the engineers' screen: it reads those files and draws live charts, so you can compare runs and spot problems early.

Animated Diagram — From Training Loop to Dashboard

Event files are in the TFRecord format you met in lesson 3 — each record is one logged value.

Section 02

Setup

📦

Install TensorBoard First

Since TensorFlow 2.21, pip install tensorflow no longer installs TensorBoard for you. Without it, tf.summary.scalar raises TBNotInstalledError. Run pip install tensorboard. (Google Colab already has it.)

pip install tensorboard

# in a notebook (Colab / Jupyter):
%load_ext tensorboard
%tensorboard --logdir logs

# in a terminal, then open http://localhost:6006
tensorboard --logdir logs

Section 03

Writing Summaries

Function	Logs	TensorBoard tab
`tf.summary.scalar(name, value, step)`	One number: loss, accuracy, learning rate	Scalars / Time Series
`tf.summary.histogram(name, tensor, step)`	Distribution of values: weights, gradients	Histograms / Distributions
`tf.summary.image(name, images, step)`	A batch of images, shape (k, h, w, c)	Images
`tf.summary.text(name, text, step)`	Notes, settings, sample predictions	Text

import tensorflow as tf
import shutil
shutil.rmtree("/tmp/logs", ignore_errors=True)

writer = tf.summary.create_file_writer("/tmp/logs/demo")
with writer.as_default():                                   # summaries go to this writer
    for step in range(5):
        tf.summary.scalar("loss", 1.0 / (step + 1), step=step)
        tf.summary.histogram("weights", tf.random.normal([200], stddev=1 + step), step=step)
    tf.summary.text("notes", "first experiment: lr=0.01", step=0)
    tf.summary.image("sample", tf.random.uniform([2, 8, 8, 3]), step=0)
writer.flush()

import os
print(os.listdir("/tmp/logs/demo"))

OUTPUT

['events.out.tfevents.1790504782.vm.4261.0.v2']

Proof: Read the Event File Back

An event file is a TFRecord file of Event protos. We can read it with the tools from lesson 3.

import glob
path = glob.glob("/tmp/logs/demo/events.*")[0]
for raw in tf.data.TFRecordDataset(path):
    event = tf.compat.v1.Event.FromString(raw.numpy())
    for v in event.summary.value:
        kind = v.metadata.plugin_data.plugin_name
        value = tf.make_ndarray(v.tensor)
        shown = round(float(value), 3) if kind == "scalars" else f"array {value.shape}"
        print(f"step {event.step}  {v.tag:8s} {kind:10s} {shown}")

OUTPUT

step 0 loss scalars 1.0 step 0 weights histograms array (30, 3) step 1 loss scalars 0.5 step 1 weights histograms array (30, 3) step 2 loss scalars 0.333 step 2 weights histograms array (30, 3) step 3 loss scalars 0.25 step 3 weights histograms array (30, 3) step 4 loss scalars 0.2 step 4 weights histograms array (30, 3) step 0 notes text array () step 0 sample images array (4,)

Section 04

Logging a Real Training Run

Use one sub-folder per run, and train / val writers inside it. TensorBoard then shows every run as a separate coloured line, ready to compare.

import numpy as np, datetime
rng = np.random.default_rng(0)
X = rng.normal(size=(512, 4)).astype("float32")
y = (X @ np.array([[1.5], [-2.0], [0.5], [1.0]], dtype="float32") + rng.normal(0, 0.3, (512, 1))).astype("float32")
X_tr, y_tr, X_va, y_va = X[:400], y[:400], X[400:], y[400:]

def run_experiment(lr, epochs=30):
    name = f"lr_{lr}"
    train_w = tf.summary.create_file_writer(f"/tmp/logs/{name}/train")
    val_w = tf.summary.create_file_writer(f"/tmp/logs/{name}/val")
    tf.random.set_seed(1)
    model = tf.keras.Sequential([tf.keras.Input((4,)), tf.keras.layers.Dense(16, activation="relu"), tf.keras.layers.Dense(1)])
    opt = tf.keras.optimizers.SGD(lr)
    ds = tf.data.Dataset.from_tensor_slices((X_tr, y_tr)).shuffle(400, seed=1).batch(32)
    for epoch in range(epochs):
        for xb, yb in ds:
            with tf.GradientTape() as tape:
                loss = tf.reduce_mean((model(xb) - yb) ** 2)
            grads = tape.gradient(loss, model.trainable_variables)
            opt.apply_gradients(zip(grads, model.trainable_variables))
        val_loss = tf.reduce_mean((model(X_va) - y_va) ** 2)
        with train_w.as_default():
            tf.summary.scalar("loss", loss, step=epoch)
            tf.summary.histogram("kernel_0", model.layers[0].kernel, step=epoch)
        with val_w.as_default():
            tf.summary.scalar("loss", val_loss, step=epoch)
    return float(val_loss)

for lr in (0.001, 0.01, 0.1):
    print(f"lr={lr:<6} final val loss {run_experiment(lr):.4f}")
print(sorted(os.listdir("/tmp/logs")))

OUTPUT

lr=0.001 final val loss 1.5138 lr=0.01 final val loss 0.1137 lr=0.1 final val loss 0.1032 ['demo', 'lr_0.001', 'lr_0.01', 'lr_0.1']

💡

Name Your Runs Well

Put the settings into the folder name (lr_0.01_bs32_drop0.2) or add a timestamp with datetime.datetime.now().strftime("%Y%m%d-%H%M%S"). Six months later you will still know which curve was which.

Section 05

Try It — A Mini TensorBoard

📈 Launch Runs, Compare, Smooth Interactive

Pick a learning rate and batch size, then press Start run. Each run streams its (simulated) loss like a real TensorBoard. Launch several and compare. Move the smoothing slider — it uses the same formula as TensorBoard. Hover the chart to read values. Switch to Histograms to watch the weights spread out during training.

learning rate batch size

smoothing: 0.60 view

Runs (click to hide)

Section 06

What to Look For

📉

Loss curves

Train and val falling together = good. Val rising while train falls = overfitting.

Scalars

💥

Spikes and NaN

Sudden jumps or a line that vanishes = learning rate too high or bad data.

Scalars

📊

Weight histograms

Weights that never change = dead layer. Weights that explode = instability.

Histograms

🎨

Sample outputs

Log a few predictions as images or text to see what the model actually does.

Images / Text

⚖️

Compare runs

One folder per experiment. Tick runs on and off to compare settings.

run folders

⏳

Log wisely

Scalars every epoch (or every N steps); histograms less often — they are bigger.

cost

Section 07

Golden Rules

📈 Logging — Rules to Remember

pip install tensorboard — TensorFlow 2.21+ no longer installs it for you.

Create a writer with tf.summary.create_file_writer(dir) and log inside with writer.as_default():.

Always pass step=. Use separate writers (folders) for train and validation.

One folder per run, with the settings in its name. Point TensorBoard at the parent folder.

Smoothing helps you see trends in noisy curves — but check the raw curve for spikes.