how does a gpt learn ?

22 Jul, 2026

begin with one unfinished sentence

Consider the short unfinished sentence

the cat sat on the ?

It looks simple to us because we read it almost in a single go. A language model does not begin with that understanding. It does not receive the sentence as meaning; it receives a sequence of token IDs.

A tokenizer first breaks the text into tokens and maps each token to an integer. The model then turns those integers into vectors, lets the vectors exchange information under a rule that hides the future, and asks the same question at every position:

What token should come next?

That one question connects the forward pass, the loss, backpropagation, and text generation. The aim of this blog is to follow that thread from beginning to end.

We will use a deliberately small example so that every object, equation, and shape remains visible.

build a tiny vocabulary

Assume the vocabulary contains the following seven tokens:

𝒱 = {[BOS], the, cat, sat, on, mat, [EOS]}

A vocabulary is like a dictionary: it maps each token to a unique integer ID. For this example, let

[BOS] \mapsto 0, the \mapsto 1, cat \mapsto 2, sat \mapsto 3, on \mapsto 4, mat \mapsto 5, [EOS] \mapsto 6

The vocabulary size is therefore

V = 7

The IDs are only labels. Token ID (2) is not mathematically closer to token ID (3) than it is to token ID (6). The model must learn useful vector representations for these IDs.

For the main example, let the model dimension be

d = 4

This means that the model represents every token position using a vector with four real-valued components.

Real language models use much larger vocabularies, sequence lengths, and model dimensions, but the same equations and tensor shapes still apply.

one sentence creates five prediction positions

Suppose the training text contains

the cat sat on the mat

Using our token-ID map, the sentence becomes

[\begin{matrix} 1 & 2 & 3 & 4 & 1 & 5 \end{matrix}]

The sentence contains six tokens:

the cat sat on the mat

To create a next-token target, every input position needs a token immediately after it. The final token has no later token inside this six-token example, so the six tokens create five prediction positions:

N = 6 - 1 = 5

The input sequence contains the first five tokens, and the target sequence contains the final five tokens:

𝐱 = [\begin{matrix} 1 & 2 & 3 & 4 & 1 \end{matrix}], 𝐲 = [\begin{matrix} 2 & 3 & 4 & 1 & 5 \end{matrix}]

More precisely,

𝐱, 𝐲 \in {0, 1, \dots, V - 1}^{N} \subset ℕ^{N},

where

ℕ_{0} = {0, 1, 2, \dots}

For this example,

𝐱, 𝐲 \in {0, 1, \dots, 6}^{5}

At position $(t)$ , the target is the token one position to the right:

y_{t} = s_{t + 1},

where $(s_{1}, \dots, s_{6})$ denotes the original six-token sequence. Equivalently,

𝐱 = [\begin{matrix} s_{1} & s_{2} & s_{3} & s_{4} & s_{5} \end{matrix}], 𝐲 = [\begin{matrix} s_{2} & s_{3} & s_{4} & s_{5} & s_{6} \end{matrix}]

The text creates its own labels. No separate answer file is required: the token after each prefix becomes the correct target for that prefix.

In words, the five training relationships are

\begin{matrix} available prefix & correct next token \\ the & cat \\ the cat & sat \\ the cat sat & on \\ the cat sat on & the \\ the cat sat on the & mat \end{matrix}

Using token IDs, the same relationships are

\begin{matrix} [1] & ⟶ & 2 \\ [4pt] [1, 2] & ⟶ & 3 \\ [4pt] [1, 2, 3] & ⟶ & 4 \\ [4pt] [1, 2, 3, 4] & ⟶ & 1 \\ [4pt] [1, 2, 3, 4, 1] & ⟶ & 5 \end{matrix}

These are not five independent word-pair examples. They are five causal views of one sequence.

pass the whole sequence through the model at once

During training, the entire input sequence

𝐱 = [\begin{matrix} 1 & 2 & 3 & 4 & 1 \end{matrix}]

is passed through the Transformer in one forward pass.

The model is causal, so the representation at each position may use only that position and the positions before it. It may not use any future position. The whole sequence is physically present in the input matrix, but every row has a different visibility pattern.

A simple binary visibility matrix makes this clear:

C = [\begin{matrix} 1 & 0 & 0 & 0 & 0 \\ 1 & 1 & 0 & 0 & 0 \\ 1 & 1 & 1 & 0 & 0 \\ 1 & 1 & 1 & 1 & 0 \\ 1 & 1 & 1 & 1 & 1 \end{matrix}]

Here,

C_{t, j} = {\begin{matrix} 1, & j \leq t, \\ 0, & j > t \end{matrix}

A $(1)$ means that position $(t)$ is allowed to use position (j), while a $(0)$ means that position $(j)$ is hidden from it.

With token labels and targets attached, the same structure is

\begin{matrix} {the}_{(1)} & {cat}_{(2)} & {sat}_{(3)} & {on}_{(4)} & {the}_{(5)} & target \\ 1 & 1 & 0 & 0 & 0 & 0 & cat \\ 2 & 1 & 1 & 0 & 0 & 0 & sat \\ 3 & 1 & 1 & 1 & 0 & 0 & on \\ 4 & 1 & 1 & 1 & 1 & 0 & the \\ 5 & 1 & 1 & 1 & 1 & 1 & mat \end{matrix}

The first output position can use only the and predicts cat. The second can use the cat and predicts sat. The final position can use the full input prefix the cat sat on the and predicts mat.

The model therefore performs all five prefix predictions together:

one sequence + different causal visibility at each row = [4pt] all next-token predictions in one pass

The matrix (C) is only a conceptual visibility matrix. The actual Transformer uses an additive causal mask inside attention, which we will write explicitly later.

turn token ids into vectors

The model cannot do useful geometry directly with arbitrary integer IDs, so it stores a learned embedding matrix

E \in ℝ^{V \times d}

With (V=7) and (d=4),

E \in ℝ^{7 \times 4}

Each row belongs to one vocabulary token, and each row contains four learned real numbers. For illustration, imagine

E = [\begin{matrix} 0.08 & - 0.11 & 0.20 & 0.04 \\ 0.12 & - 0.34 & 0.56 & 0.78 \\ 0.91 & 0.23 & - 0.45 & 0.67 \\ - 0.33 & 0.88 & 0.12 & - 0.56 \\ 0.45 & - 0.67 & 0.34 & 0.11 \\ 0.22 & 0.55 & - 0.78 & 0.33 \\ - 0.14 & 0.29 & 0.41 & - 0.62 \end{matrix}]

These numbers are illustrative. In a real model, they are learned during training.

Looking up the rows indexed by

𝐱 = [\begin{matrix} 1 & 2 & 3 & 4 & 1 \end{matrix}]

gives

X_{t o k} = E [𝐱] = [\begin{matrix} 0.12 & - 0.34 & 0.56 & 0.78 \\ 0.91 & 0.23 & - 0.45 & 0.67 \\ - 0.33 & 0.88 & 0.12 & - 0.56 \\ 0.45 & - 0.67 & 0.34 & 0.11 \\ 0.12 & - 0.34 & 0.56 & 0.78 \end{matrix}]

Its shape is

X_{t o k} \in ℝ^{N \times d} = ℝ^{5 \times 4}

There is one row for every token position and one column for every learned feature. The rows have not been merged into one sentence vector. The model carries all five positions at the same time.

The token the occurs twice, so its raw token-embedding row appears twice. The model must still distinguish the first occurrence from the second. Position information is therefore added or otherwise introduced.

In a simple additive description,

X^{(0)} = X_{t o k} + P,

where

P \in ℝ^{N \times d}

Because both matrices have the same shape,

(N, d) + (N, d) = (N, d)

and therefore

X^{(0)} \in ℝ^{N \times d}

Position information changes the values, not the shape. Modern models may use learned positions, sinusoidal positions, rotary position embeddings, or relative-position methods. The central fact remains the same: there is one $(d)$ -dimensional state for each of the (N) positions.

one sequence, many changing viewpoints

The hidden-state shape stays the same through the Transformer.

If $(X^{(ℓ)})$ enters block $(ℓ)$ , then the block returns another matrix with the same shape:

X^{(ℓ)} \in ℝ^{N \times d} ⟶ X^{(ℓ + 1)} \in ℝ^{N \times d}

A common pre-normalization block can be written schematically as

U^{(ℓ)} & = X^{(ℓ)} + MHA (LN (X^{(ℓ)})), X^{(ℓ + 1)} & = U^{(ℓ)} + MLP (LN (U^{(ℓ)}))

Exact normalization details vary across architectures, but every residual addition requires matching shapes. Both the attention branch and the feature-processing branch therefore return $(N \times d)$ matrices.

If the shape does not change, what does change?

The meaning carried by each row changes.

At the beginning, a row mostly represents one token at one position. After many Transformer blocks, the same row can represent that token together with information gathered from its permitted prefix.

After $(L)$ blocks, let

H = X^{(L)} = [\begin{matrix} h_{1}^{⊤} \\ h_{2}^{⊤} \\ h_{3}^{⊤} \\ h_{4}^{⊤} \\ h_{5}^{⊤} \end{matrix}] \in ℝ^{5 \times d}

Conceptually, the rows are allowed to encode

h_{1} ⇝ the, h_{2} ⇝ the cat, h_{3} ⇝ the cat sat, h_{4} ⇝ the cat sat on, h_{5} ⇝ the cat sat on the

The vectors do not literally contain those strings. This notation describes the information each row is permitted to depend on.

how the future stays hidden

If all five positions are present in the same matrix, why can the row for cat not look ahead and copy information from sat?

This is the purpose of causal self-attention.

For one attention head, start with

X \in ℝ^{N \times d}

The model forms queries, keys, and values:

Q = X W_{Q}, K = X W_{K}, V_{a} = X W_{V},

where

W_{Q}, W_{K}, W_{V} \in ℝ^{d \times d_{h}}

The symbol (V_{!a}) denotes the attention-value matrix. It is different from the vocabulary size (V).

The projected matrices have shape

Q, K, V_{a} \in ℝ^{N \times d_{h}}

A useful interpretation is

each query row describes what the current position is looking for;
each key row describes what a position can be matched by;
each value row contains the information that position can contribute.

All pairwise attention scores are computed together:

S = \frac{Q K^{⊤}}{\sqrt{d_{h}}}

The shape calculation is

(N, d_{h}) (d_{h}, N) = (N, N),

S \in ℝ^{N \times N}

Entry (S_{t,j}) measures how strongly position (t) matches position (j) before masking and normalization.

The additive causal mask is

M_{t, j} = {\begin{matrix} 0, & j \leq t, \\ - \infty, & j > t \end{matrix}

For five positions,

M = [\begin{matrix} 0 & - \infty & - \infty & - \infty & - \infty \\ 0 & 0 & - \infty & - \infty & - \infty \\ 0 & 0 & 0 & - \infty & - \infty \\ 0 & 0 & 0 & 0 & - \infty \\ 0 & 0 & 0 & 0 & 0 \end{matrix}]

The attention weights are computed row by row:

A = softmax (\frac{Q K^{⊤}}{\sqrt{d_{h}}} + M)

Because

e^{- \infty} = 0,

every forbidden future position receives exactly zero attention weight. The resulting pattern is

A = [\begin{matrix} * & 0 & 0 & 0 & 0 \\ * & * & 0 & 0 & 0 \\ * & * & * & 0 & 0 \\ * & * & * & * & 0 \\ * & * & * & * & * \end{matrix}],

where every (*) is a nonnegative value and every row sums to (1).

The attention output is

Z_{a} = A V_{a}

For row (t),

z_{a, t} = \sum_{j = 1}^{t} A_{t, j} v_{j}

The upper limit is (t), not (N). Future positions do not appear in the sum. This is the precise mathematical statement that the future is hidden.

The sequence length does not grow inside the forward pass. The model already has a fixed (N\times N) attention grid. What changes from row to row is the amount of context that is visible.

In short,

(N, d) ⟶ (N, N) attention relations ⟶ (N, d)

Real Transformers use multiple heads, allowing different relationships to be represented in parallel. After the heads are joined and projected, the output returns to shape (N\times d).

from hidden states to next-token probabilities

After the final Transformer block, we have

H \in ℝ^{N \times d}

Each row must now become a score for every vocabulary token. The model uses an output projection

W_{U} \in ℝ^{d \times V}

and usually a bias

b_{U} \in ℝ^{V}

For each position,

z_{t} = h_{t} W_{U} + b_{U}, z_{t} \in ℝ^{V}

Applying the same projection to all rows gives

Z = H W_{U} + b_{U},

where the bias is broadcast across the (N) rows.

The shape calculation is

(N, d) (d, V) = (N, V)

With (N=5), (d=4), and (V=7),

(5, 4) (4, 7) = (5, 7)

Z \in ℝ^{5 \times 7}

There are five input positions, and every position receives seven vocabulary scores.

Suppose the logits are

Z = [\begin{matrix} - 1.2 & 0.3 & 2.0 & 0.1 & - 0.5 & 0.4 & - 1.0 \\ - 0.8 & - 0.3 & 0.5 & 2.5 & 0.7 & 0.1 & - 1.2 \\ - 1.5 & 0.2 & - 0.1 & 0.4 & 3.0 & - 0.2 & - 1.0 \\ - 2.0 & 2.8 & 0.1 & - 0.3 & 0.5 & - 0.5 & - 1.2 \\ - 2.0 & - 0.5 & 0.1 & - 0.3 & 0.5 & 2.8 & - 1.0 \end{matrix}]

with columns ordered as

[[BOS], the, cat, sat, on, mat, [EOS]]

A logit is an unrestricted real-valued score. Softmax converts each row into a probability distribution:

p_{t, v} = \frac{e^{z_{t, v}}}{\sum_{u = 0}^{V - 1} e^{z_{t, u}}}

Softmax is applied separately to every row, so

\sum_{v = 0}^{V - 1} p_{t, v} = 1 for every position t

For the logits above, the approximate probability matrix is

P \approx [\begin{matrix} 0.0239 & 0.1070 & 0.5859 & 0.0876 & 0.0481 & 0.1183 & 0.0292 \\ 0.0244 & 0.0402 & 0.0894 & 0.6606 & 0.1092 & 0.0599 & 0.0163 \\ 0.0089 & 0.0486 & 0.0360 & 0.0594 & 0.7998 & 0.0326 & 0.0146 \\ 0.0064 & 0.7837 & 0.0527 & 0.0353 & 0.0786 & 0.0289 & 0.0144 \\ 0.0064 & 0.0288 & 0.0525 & 0.0352 & 0.0783 & 0.7813 & 0.0175 \end{matrix}]

The bold entries are the probabilities assigned to the correct next tokens:

p_{θ} (cat ∣ the) & \approx 0.5859, p_{θ} (sat ∣ the cat) & \approx 0.6606, p_{θ} (on ∣ the cat sat) & \approx 0.7998, p_{θ} (the ∣ the cat sat on) & \approx 0.7837, p_{θ} (mat ∣ the cat sat on the) & \approx 0.7813

The network has produced five different next-token distributions in one pass. Row (1) answers “what follows the?” Row (5) answers “what follows the cat sat on the?”

During generation, only the newest row is needed. During training, every row has a known target, so every row can contribute to the loss.

why cross-entropy is the right loss

At position (t), let

p_{t} = [\begin{matrix} p_{t, 0} & p_{t, 1} & \dots & p_{t, V - 1} \end{matrix}]

be the model's predicted distribution over the vocabulary.

Let (y_t) be the correct token ID, and let (q_t) be its one-hot target distribution:

q_{t, v} = {\begin{matrix} 1, & v = y_{t}, \\ 0, & v \neq y_{t} \end{matrix}

The categorical cross-entropy is

ℓ_{t} = - \sum_{v = 0}^{V - 1} q_{t, v} \log p_{t, v}

Because every incorrect entry of (q_t) is zero, all terms disappear except the correct one:

ℓ_{t} = - \log p_{t, y_{t}} .

This gives a simple interpretation:

cross-entropy loss = surprise assigned to the correct token

A high probability for the correct token gives a small loss:

- \log (1) = 0 .

A low probability gives a large loss:

{lim}_{p \to 0^{+}} - \log p = + \infty

Some reference values make the scale intuitive:

\begin{matrix} p_{correct} & - \log p_{correct} \\ 0.90 & 0.105 \\ 0.50 & 0.693 \\ 0.10 & 2.303 \\ 0.01 & 4.605 \end{matrix}

The function (-\log p) decreases continuously as (p) increases. Therefore, for a one-hot target, minimizing cross-entropy is exactly the same as pushing the probability of the correct token upward.

There is also a sequence-level reason for the logarithm.

The causal model assigns the correct continuation the probability

& p_{θ} (cat sat on the mat ∣ the) [4pt] & = p_{θ} (cat ∣ the) \cdot p_{θ} (sat ∣ the cat) \cdot p_{θ} (on ∣ the cat sat) & \cdot p_{θ} (the ∣ the cat sat on) \cdot p_{θ} (mat ∣ the cat sat on the)

This is the chain rule of probability. The capital pi notation is a compact way to write the same product. If the original sequence is

𝐬 = [\begin{matrix} s_{1} & s_{2} & \dots & s_{N + 1} \end{matrix}],

then

p_{θ} (s_{2}, \dots, s_{N + 1} ∣ s_{1}) = \prod_{t = 1}^{N} p_{θ} (s_{t + 1} ∣ s_{\leq t}) = \prod_{t = 1}^{N} p_{θ} (y_{t} ∣ x_{\leq t})

Here,

\prod_{t = 1}^{N} a_{t} = a_{1} a_{2} \dots a_{N}

Taking the negative logarithm turns the product into a sum:

- \log p_{θ} (𝐲 ∣ 𝐱) & = - \log \prod_{t = 1}^{N} p_{θ} (y_{t} ∣ x_{\leq t}) [4pt] & = \sum_{t = 1}^{N} - \log p_{θ} (y_{t} ∣ x_{\leq t}) [4pt] & = \sum_{t = 1}^{N} ℓ_{t}

Therefore,

sum of token cross-entropies = negative log-probability of the correct continuation .

This is the key intuitive proof.

The model is not being trained on unrelated word pairs. Maximizing the probability of the entire correct continuation is equivalent to minimizing the sum of its next-token cross-entropies.

The mean training loss is usually

ℒ = \frac{1}{N} \sum_{t = 1}^{N} ℓ_{t}

Dividing by (N) changes the scale but not the best model parameters for a fixed sequence length. It also makes losses and gradient magnitudes more comparable across examples containing different numbers of valid target tokens.

For the five correct probabilities above,

ℓ_{1} & = - \log (0.5859) \approx 0.5346, ℓ_{2} & = - \log (0.6606) \approx 0.4146, ℓ_{3} & = - \log (0.7998) \approx 0.2234, ℓ_{4} & = - \log (0.7837) \approx 0.2437, ℓ_{5} & = - \log (0.7813) \approx 0.2469

The per-token loss vector is

ℓ \approx [\begin{matrix} 0.5346 & 0.4146 & 0.2234 & 0.2437 & 0.2469 \end{matrix}] \in ℝ^{5}

The mean is

ℒ = \frac{0.5346 + 0.4146 + 0.2234 + 0.2437 + 0.2469}{5} \approx 0.3326

The product of the five correct probabilities is approximately

p_{sequence} \approx 0.1895,

and indeed,

- \log (0.1895) \approx 1.6632 = \sum_{t = 1}^{5} ℓ_{t} .

Dividing by five gives the same mean:

\frac{1.6632}{5} \approx 0.3326

For a more general target distribution (q), cross-entropy also has the identity

H (q, p) = H (q) + D_{K L} (q ‖ p)

Because

D_{K L} (q ‖ p) \geq 0,

cross-entropy is minimized when the predicted distribution (p) matches the target distribution (q). For ordinary next-token training, (q) is one-hot, so this reduces to assigning as much probability as possible to the correct token.

In practice, software computes cross-entropy directly from logits using a numerically stable log-softmax operation:

ℓ_{t} = - z_{t, y_{t}} + \log (\sum_{v = 0}^{V - 1} e^{z_{t, v}})

This is algebraically equal to (-\log p_{t,y_t}), but it avoids numerical problems caused by explicitly exponentiating very large or very small values.

how one scalar changes every weight

The scalar loss was produced through a chain of differentiable operations:

token IDs ⟶ embeddings ⟶ hidden states ⟶ logits ⟶ cross-entropy ⟶ ℒ

Every learnable parameter that influenced the logits also influenced the loss. Autograd records these operations and applies the chain rule backward through the graph.

For a parameter (\theta_i), the gradient

\frac{\partial ℒ}{\partial θ_{i}}

answers the local question

How would the loss change if θ_{i} changed slightly?

Softmax cross-entropy has a particularly clear gradient. If (e_{y_t}) is the one-hot vector for the correct token, then

\frac{\partial ℓ_{t}}{\partial z_{t}} = p_{t} - e_{y_{t}}

For the correct token,

\frac{\partial ℓ_{t}}{\partial z_{t, y_{t}}} = p_{t, y_{t}} - 1,

which is negative unless the correct probability is already (1). Gradient descent subtracts the gradient, so it tends to increase the correct token's logit.

For an incorrect token (v),

\frac{\partial ℓ_{t}}{\partial z_{t, v}} = p_{t, v} > 0

Gradient descent therefore tends to decrease that incorrect logit. An incorrect token that received more probability receives a stronger downward correction.

All positions use the same model parameters. If

ℒ = \frac{1}{N} \sum_{t = 1}^{N} ℓ_{t},

then differentiation is linear:

\nabla_{θ} ℒ = \frac{1}{N} \sum_{t = 1}^{N} \nabla_{θ} ℓ_{t}

This is why the model does not need one backward pass per token. We compute all token losses, reduce them to one scalar, and call backward once. The resulting gradient already contains the contribution from every supervised position.

A basic gradient-descent update is

θ_{k + 1} = θ_{k} - η \nabla_{θ} ℒ (θ_{k}),

where (\eta) is the learning rate.

Modern optimizers such as Adam use additional state and rescaling, but the central action is the same: change the parameters in a direction expected to reduce future loss.

One training step is therefore

forward pass ⟶ scalar loss ⟶ backward pass ⟶ parameter update

Repeated over many sequences, this process changes the embedding table, attention projections, MLP weights, normalization parameters, and output projection so that correct continuations receive more probability.

why training is parallel but generation is sequential

Training and generation use the same causal next-token model, but they use its output differently.

during training

The correct continuation is already known.

For the input

[the, cat, sat, on, the]

the shifted target is

[cat, sat, on, the, mat]

The causal mask prevents each row from seeing future input tokens, while the target sequence provides the correct answer for every output row. All five positions can therefore be trained together:

Z \in ℝ^{N \times V} ⟶ N supervised next-token decisions

This is commonly called teacher forcing: every position receives the true earlier tokens from the dataset rather than earlier tokens sampled from the model.

during generation

Suppose the prompt is

[the, cat]

A forward pass produces one logit row per prompt position:

Z \in ℝ^{2 \times V}

The first row predicts what follows the, but the prompt already contains cat. Only the final row predicts the next unseen token. Generation therefore uses

z_{N, :}

The next token may be chosen greedily,

{\hat{x}}_{N + 1} = \arg {max}_{v \in 𝒱} z_{N, v} .

or sampled from a temperature-scaled distribution,

{\hat{x}}_{N + 1} ~ Categorical (softmax (\frac{z_{N}}{τ})),

where $(τ > 0)$ is the temperature.

If the model chooses sat, it appends that token:

[the, cat] ⟶ [the, cat, sat]

The process then repeats:

\begin{matrix} [the, cat] & ⟶ & sat \\ [4pt] [the, cat, sat] & ⟶ & on \\ [4pt] [the, cat, sat, on] & ⟶ & the \\ [4pt] [the, cat, sat, on, the] & ⟶ & mat \end{matrix}

Generation is sequential because the next prediction depends on the token just generated:

{\hat{x}}_{t + 1} ~ p_{θ} (\cdot ∣ x_{\leq t})

Before $({\hat{x}}_{t + 1})$ exists, it cannot be part of the context used to produce $({\hat{x}}_{t + 2})$ .

A key-value cache can reuse attention keys and values computed for earlier tokens, making generation much faster. It does not remove the one-new-token-at-a-time dependency.

beginning and end tokens

The main example treats the first the as given context and predicts the five tokens after it.

To assign a probability to the first token too, place $[BOS]$ before the sentence. To teach the model when to stop, place $[EOS]$ after it:

[BOS] the cat sat on the mat [EOS]

Then the input and target could be

𝐱_{b o u n d a r y} = [\begin{matrix} 0 & 1 & 2 & 3 & 4 & 1 & 5 \end{matrix}]

𝐲_{b o u n d a r y} = [\begin{matrix} 1 & 2 & 3 & 4 & 1 & 5 & 6 \end{matrix}]

This version has seven prediction positions. It includes both

p_{θ} (the ∣ [BOS)

and

p_{θ} ([EOS] ∣ the cat sat on the mat)

The earlier $(N = 5)$ example is still correct; it simply studies the continuation after the initial the and stops at mat.

attention masks and loss masks are different

A causal attention mask answers

Which input positions may influence this hidden state?

A loss mask answers

Which output positions count toward the training objective?

These are different operations.

For ordinary language-model pretraining, most real token positions contribute to the loss. Padding positions are normally excluded.

In supervised fine-tuning, the entire prompt and response may enter the model, while only response positions contribute directly to the loss. If $(m_{t} \in {0, 1})$ marks supervised positions, then

ℒ_{m a s k e d} = \frac{\sum_{t} m_{t} ℓ_{t}}{\sum_{t} m_{t}}

A prompt token with $(m_{t} = 0)$ is still present and can influence later answer states through attention. It is merely excluded from the final loss average.

the whole picture without the clutter

For one sequence, the complete path is

𝐱 \in {0, \dots, V - 1}^{N} \overset{embedding and position}{\to} X^{(0)} \in ℝ^{N \times d} \overset{Transformer blocks}{\to} H \in ℝ^{N \times d}

followed by

H \overset{output projection}{\to} Z \in ℝ^{N \times V} \overset{cross-entropy}{\to} ℓ \in ℝ^{N} \overset{mean}{\to} ℒ \in ℝ

With a batch of $(B)$ sequences, a head count of $(H_{h e a d s})$ , and a common stored sequence length $(N)$ ,

token IDs \in {0, \dots, V - 1}^{B \times N}, hidden states \in ℝ^{B \times N \times d}, attention scores \in ℝ^{B \times H_{h e a d s} \times N \times N}, logits \in ℝ^{B \times N \times V}, token losses \in ℝ^{B \times N}, reduced loss \in ℝ .

If $(m_{b, t})$ marks valid supervised targets, the batch loss is

ℒ = \frac{\sum_{b = 1}^{B} \sum_{t = 1}^{N} m_{b, t} ℓ_{b, t}}{\sum_{b = 1}^{B} \sum_{t = 1}^{N} m_{b, t}}

The central facts are now compact:

The model keeps one hidden row per token position:

$(N, d) ⟶ (N, d)$
The whole input is processed at once, but causal masking hides future columns from every row:

M_{t, j} = {\begin{matrix} 0, & j \leq t, \\ - \infty, & j > t . \end{matrix}

Every final hidden row becomes a complete vocabulary distribution:

$(N, d) (d, V) = (N, V)$
The target sequence is the original sequence shifted by one token:

$y_{t} = s_{t + 1}$
Token cross-entropies are the additive pieces of one continuation-level negative log-probability:

\sum_{t = 1}^{N} ℓ_{t} = - \log \prod_{t = 1}^{N} p_{θ} (y_{t} ∣ x_{\leq t}) .

One backward pass includes the learning signal from every supervised position:

\nabla_{θ} ℒ = \frac{1}{N} \sum_{t = 1}^{N} \nabla_{θ} ℓ_{t}

Training uses every valid output row because every target is known. Generation uses the newest row because only the next unseen token matters.

As one story, the process is very orderly. The model keeps one evolving vector for every position. Each vector may gather information only from its past. Each final vector proposes a distribution over what should come next. The correct next tokens are scored with cross-entropy, the token losses become one scalar, and that scalar sends a learning signal backward through the entire network.