tf.Variable vs tf.Tensor in TensorFlow

Section 01

The Story — A Printed Page and a Whiteboard

📖 Real World Analogy

You Cannot Erase a Printed Page

A printed page never changes. If you want different text, you print a new page. That is a tf.Tensor.

A whiteboard stays on the wall. You wipe it and write new numbers on the same board, again and again. That is a tf.Variable.

A neural network needs a whiteboard for its weights. During training, the weights change thousands of times. We want to update them in place, not make a new copy each time.

So far you have used tensors. Tensors are immutable: once made, their values never change. In this lesson you will meet tf.Variable, the mutable container that holds a model's weights. This is the first step to training any model.

📄 tf.Tensor — the printed page

Values cannot change

Every op makes a new tensor

Used for data: inputs, outputs, activations

Gradient tape does not watch it by default

📝 tf.Variable — the whiteboard

Values can change in place

Update with assign, assign_add, assign_sub

Used for model state: weights, biases, counters

Gradient tape watches it automatically

Section 02

Creating a tf.Variable

You create a variable from a starting value. The value can be a number, a list, a NumPy array, or a tensor.

import tensorflow as tf

w = tf.Variable([[1.0, 2.0],
                 [3.0, 4.0]], name="weights")
b = tf.Variable(0.5, name="bias")

print(w)
print("name     :", w.name)
print("shape    :", w.shape)
print("dtype    :", w.dtype)
print("trainable:", w.trainable)

OUTPUT

<tf.Variable 'weights:0' shape=(2, 2) dtype=float32, numpy= array([[1., 2.], [3., 4.]], dtype=float32)> name : weights:0 shape : (2, 2) dtype : <dtype: 'float32'> trainable: True

A variable has the same shape and dtype rules as a tensor. It also has a name (useful when saving models) and a trainable flag.

Variables Work Like Tensors in Maths

You can use a variable anywhere you use a tensor. The result of any maths op is a plain tensor, not a variable.

v = tf.Variable([1.0, 2.0, 3.0])

result = v * 10 + 1
print(result)
print("v type     :", type(v).__name__)
print("result type:", type(result).__name__)
print("v unchanged:", v.numpy())

OUTPUT

tf.Tensor([11. 21. 31.], shape=(3,), dtype=float32) v type : ResourceVariable result type: EagerTensor v unchanged: [1. 2. 3.]

Section 03

Changing a Variable — assign, assign_add, assign_sub

Method	Does	Same as
`v.assign(x)`	Replace all values	`v = x` (but in place)
`v.assign_add(x)`	Add to values	`v += x`
`v.assign_sub(x)`	Subtract from values	`v -= x` — used in gradient descent
`v[i].assign(x)`	Change part of the variable	slice update

v = tf.Variable([1.0, 2.0, 3.0])

v.assign([10.0, 20.0, 30.0]);  print("assign    :", v.numpy())
v.assign_add([1.0, 1.0, 1.0]); print("assign_add:", v.numpy())
v.assign_sub([5.0, 5.0, 5.0]); print("assign_sub:", v.numpy())
v[0].assign(-1.0);             print("v[0]      :", v.numpy())

OUTPUT

assign : [10. 20. 30.] assign_add: [11. 21. 31.] assign_sub: [ 6. 16. 26.] v[0] : [-1. 16. 26.]

Shape and dtype Are Fixed

You can change the values, but not the shape or the dtype. The whiteboard has a fixed size.

v = tf.Variable([1, 2, 3])        # int32, shape (3,)

try:
    v.assign([1, 2])
except ValueError as e:
    print("ValueError:", str(e).split("Shape mismatch.")[-1])

try:
    v.assign([1.5, 2.0, 3.0])
except TypeError as e:
    print("TypeError :", e)

OUTPUT

ValueError: The variable shape (3,), and the assigned value shape (2,) are incompatible. TypeError : Cannot convert [1.5, 2.0, 3.0] to EagerTensor of dtype int32

⚠️

Use Float Variables for Weights

tf.Variable(3) is an int32 variable. Gradients do not flow through integers. Always write tf.Variable(3.0) for anything you want to train.

Section 04

Try It — Tensor vs Variable Lab

🧪 Same Operation, Two Containers Interactive

Both start as [1, 2, 3]. Pick an operation and a value, then press Run. Watch the memory boxes. A new box means a new object was created. A flashing box means it was changed in place. Try a wrong shape like 5 or 1, 2 with assign.

Operation Value x

📄 t = tf.constant([1, 2, 3])

📝 v = tf.Variable([1, 2, 3])

# Your code will appear here

👀

The Hidden Trap: v = v + x

Choose obj = obj + x on the Variable side. The name v now points to a new tensor. The variable is lost, and so is its link to the gradient tape. In a training loop this silently stops learning. Always use v.assign_sub(...) to update weights.

Section 05

Where Variables Live in a Real Model

Every Keras layer creates its weights as variables. You can list them.

layer = tf.keras.layers.Dense(3)
layer.build(input_shape=(None, 4))     # 4 inputs -> 3 units

for var in layer.trainable_variables:
    print(f"{var.path:18s} shape={var.shape}  dtype={var.dtype}")

OUTPUT

dense/kernel shape=(4, 3) dtype=float32 dense/bias shape=(3,) dtype=float32

Animated Diagram — Data Flows, Weights Stay and Update

Input batches are new tensors every step. The weights are one variable that stays in memory and is nudged after every step.

Non-trainable Variables

Some state must change but must not be trained, such as a step counter or the running mean in BatchNorm. Use trainable=False.

step = tf.Variable(0, trainable=False, name="step")
for _ in range(3):
    step.assign_add(1)
print("steps done:", step.numpy(), "| trainable:", step.trainable)

OUTPUT

steps done: 3 | trainable: False

Section 06

Quick Comparison

Feature	tf.Tensor	tf.Variable
Create	`tf.constant(...)`, any op	`tf.Variable(initial_value)`
Change values	No	Yes — assign*
Change shape / dtype	No	No
Watched by GradientTape	Only with tape.watch()	Yes, if trainable
Saved in checkpoints	No	Yes
Typical use	Data, activations, results	Weights, biases, counters

Section 07

Golden Rules

📝 Variables — Rules to Remember

Use tensors for data. Use variables for anything that must be learned or remembered.

Update variables with assign, assign_add or assign_sub. Never with v = v - x.

A variable's shape and dtype are fixed when you create it.

Create trainable variables as floats: tf.Variable(3.0), not tf.Variable(3).

Maths on a variable returns a plain tensor. The variable itself only changes through assign methods.