Understanding Deep Learning
One of the most fundamental questions in modern machine learning is why deep learning works so remarkably well across such different domains. Problems such as protein structure prediction, autonomous driving, language generation, image recognition, and playing complex games appear to have little in common. Yet neural networks, trained using broadly similar principles, have achieved extraordinary results across all of them.
Historically, solving these problems often required designing specialized algorithms based on our understanding of each domain. A chess program might rely on handcrafted evaluation functions, while a computer vision system might depend on manually engineered image features. Deep learning introduced a different approach: rather than explicitly specifying every rule needed to solve a problem, we construct a parameterized model and allow data and optimization to shape its behavior.
The underlying ambition is powerful. Perhaps a sufficiently expressive neural network, given appropriate data, a suitable training objective, and an effective optimization procedure, can learn useful functions across an enormous variety of domains.
This is not a guarantee that any arbitrary problem can be learned. Rather, it is a hypothesis about the generality of learning from data, and the empirical success of deep learning suggests that this approach is much more broadly applicable than we might initially have expected.
But this immediately raises a deeper question: Why does optimizing enormous neural networks work so well, and can we predict the circumstances under which it will succeed or fail?
To understand this question, we need to distinguish three fundamentally different properties of neural networks.
Expressivity concerns whether a neural network can represent the desired function.
Optimization concerns whether a training algorithm can actually discover parameters that approximate that function.
Generalization concerns whether the learned function performs well on inputs that were not encountered during training.
These properties are related, but none automatically guarantees the others.
Consider training a neural network to recognize cats. A sufficiently expressive network may be capable of representing an accurate image classifier. However, this says little about whether gradient descent will discover that classifier, or whether the learned classifier will recognize cats in unfamiliar photographs.
This distinction is central to understanding deep learning. The fact that a model can represent a solution does not establish that training will find it, and finding a solution that performs well on training data does not guarantee that it will generalize.
Understanding the Algorithm Is Not the Same as Understanding Its Behavior
The basic mathematics of neural network training is relatively straightforward. Given parameters , a learning rate , and a loss function , gradient descent updates the parameters according to
We compute the gradient of the loss with respect to the parameters and move in a direction intended to reduce that loss.
Although we understand this update rule, it does not automatically tell us what internal representations the network will discover, why particular patterns emerge during training, why one architecture generalizes better than another, or why increasing model scale can produce unexpected capabilities.
In other words, understanding the mechanics of training does not imply that we possess a complete predictive theory of trained neural networks.
A neural network also does not necessarily learn the function we intended it to learn. Instead, training favors solutions determined by the data, architecture, objective, optimization procedure, and their combined inductive biases. Sometimes these solutions use statistical shortcuts that happen to reduce the training loss without capturing the relationships we actually care about.
This becomes particularly important when considering architectural choices.
Imagine two neural networks trained on the same task. Network A contains four layers, while Network B contains forty layers. At first glance, we might expect Network B to perform better because greater depth can provide additional representational capacity.
However, representation and optimization are competing considerations. The deeper network may be capable of representing more complicated functions while simultaneously being harder to optimize. Its gradients, activation magnitudes, and sensitivity to hyperparameters may behave differently.
Consequently, greater expressive capacity does not guarantee better performance.
An architectural modification as simple as a residual connection can dramatically change this situation:
Instead of requiring every layer to construct an entirely new representation, a residual block allows it to learn a modification to the existing representation. The identity path also provides a direct route for information and gradients to propagate through the network.
Without residual connections, each layer must produce the representation consumed by the next layer. With residual connections, the network can preserve the existing representation while learning useful corrections.
This can make deep networks considerably easier to optimize, even though the task and training data remain unchanged.
The broader point is that neural networks cannot be understood by examining expressive capacity alone. The way components interact during optimization is just as important as the functions those components can theoretically represent.
This also helps explain the phrase unreasonable effectiveness of deep learning. Its empirical success has repeatedly exceeded what simple theoretical intuitions might have predicted.
For example, classical intuitions about model complexity might lead us to expect extremely overparameterized networks to overfit severely. Yet large neural networks can sometimes generalize remarkably well, even when they possess sufficient capacity to memorize their training examples.
This does not make classical learning theory irrelevant. It means that simple arguments based on parameter count or capacity alone are insufficient to explain the behavior of modern neural networks.
The same difficulty arises when considering optimization.
Suppose we train two otherwise identical models with learning rates
It is tempting to assume that Model B will learn ten times faster. But a larger learning rate may produce faster convergence, oscillation, instability, or outright divergence.
Furthermore, changing the architecture can change which learning rates are effective. A learning rate that works well for one architecture might produce poor results for another.
Thus, it is not enough to understand architectural components and hyperparameters independently. We need to understand their interactions.
This introduces an important distinction between two forms of scientific understanding.
Explanatory understanding allows us to explain an experimental result after observing it. Predictive understanding allows us to anticipate the result of a new experiment before performing it.
The latter is considerably more demanding.
Suppose we observe the following validation losses:
| Architecture | Learning rate | Validation loss |
|---|---|---|
| A | 0.001 | 2.5 |
| B | 0.001 | 3.2 |
From this experiment, we can conclude that A performs better under the tested conditions. But suppose we subsequently tune B's learning rate and obtain a validation loss of 2.3.
Our original observation remains correct, yet the broader conclusion that A is the better architecture is no longer supported.
This illustrates why empirical intuitions can be brittle. What appears to be an architectural advantage may actually depend on initialization, optimization settings, training budget, or the interaction between them.
We should therefore distinguish observations, hypotheses, and general principles.
An observation might be that A trained better than B in a particular experiment. A hypothesis might be that B performed worse because its optimization was unstable. A general principle would specify the conditions under which B's architecture is expected to produce instability.
Each statement requires progressively stronger evidence.
Training Dynamics, Normalization, and the Limits of Intuition
Normalization provides a useful example of how understanding individual mathematical operations differs from predicting an entire training process.
Consider Layer Normalization:
where is the mean across the normalized features, is their variance, and is a small constant used for numerical stability. In practice, LayerNorm also includes learned scale and shift parameters.
Suppose we have two input vectors:
Their means are 20 and 200, while their standard deviations are approximately 8.165 and 81.65, respectively.
Ignoring , normalization transforms both vectors into approximately
The absolute magnitudes differ by a factor of ten, yet their normalized representations are identical.
More generally, ignoring the numerical-stability constant, LayerNorm is invariant to positive scaling and uniform shifting:
This property illustrates what normalization accomplishes: it removes certain variations in scale and offset while preserving the relative pattern of features within a normalized group.
Why might this be useful for optimization?
Consider a simple neuron:
Applying the chain rule gives
The input magnitude directly contributes to the gradient. Increasing can therefore increase the magnitude of the gradient, but it also changes , which changes the derivative of the activation function.
For a sigmoid activation, sufficiently large positive or negative inputs can cause saturation, making the derivative approach zero. For ReLU, the derivative is one for positive inputs and zero for negative inputs, away from the origin.
Consequently, large activations do not always produce large gradients. Depending on the activation function and the surrounding computation, they may contribute to exploding gradients, vanishing gradients, or other changes in training dynamics.
The deeper issue is that activation scale affects how the network responds to parameter changes and how gradients propagate through its layers.
Normalization can make some aspects of this behavior more predictable, but it does not simply make gradients smaller or guarantee stable training.
It also introduces a tradeoff. If two inputs differ only by a positive scale factor, LayerNorm can map them to the same normalized representation. When absolute magnitude carries useful information, this invariance may remove information that the model would otherwise exploit.
Thus, normalization is not automatically beneficial in every situation.
Its placement also matters. Moving normalization from after a residual block to before it changes the computational path through which information and gradients flow. Even when both designs contain the same mathematical components, their optimization behavior can be substantially different.
This is precisely where predictive understanding becomes difficult. Knowing that normalization controls activation scale is not sufficient to predict how changing its position will affect convergence, stability, or final loss.
To investigate such questions, we need controlled experiments.
We might first compare models with and without normalization under identical training settings. We would then tune the learning rate separately for each architecture, because the optimal configuration might change. In addition to final validation loss, we would examine activation norms, gradient norms, convergence behavior, and stability across layers. Repeating these experiments across several random seeds would help determine whether the observed difference is reliable.
Even after obtaining reproducible results, however, we must be careful about what we claim to understand.
Suppose replacing ReLU with SwiGLU consistently improves a Transformer across several model sizes, learning rates, and random seeds. We have established that the improvement is robust under the tested conditions.
But have we established why SwiGLU works better?
Not necessarily.
SwiGLU differs from a standard ReLU feedforward network in several ways, including its activation function, multiplicative gating, and parameterization. Any of these differences, or their interactions, could contribute to the result.
A more informative experiment would compare four configurations: an ordinary ReLU network, an ordinary SiLU network, a ReLU-based gated network, and SwiGLU. We would control the parameter budget, data, compute, and tuning procedure as carefully as possible.
If both gated models outperform their ungated counterparts, that supports the hypothesis that gating contributes to the improvement. If SiLU-based models perform better regardless of gating, the activation function may be important. If only SwiGLU performs well, the interaction between its activation function and gating mechanism may matter.
Even these experiments do not automatically reveal the complete internal mechanism. They provide evidence about which components contribute under specific conditions.
The scientific process is therefore iterative:
A useful theory should do more than explain results already observed. It should generate predictions about experiments that have not yet been performed.
This distinction is particularly important because neural network optimization is generally non-convex.
In an idealized convex optimization problem, the loss landscape resembles a smooth bowl, and appropriate optimization methods can move toward a global minimum. Neural networks instead have complicated parameter spaces in which initialization, learning rates, architecture, and data ordering can influence the solution found by training.
However, non-convexity does not automatically imply that optimization will fail. Deep neural networks are frequently optimized successfully despite these complications.
The real puzzle is understanding why gradient-based methods can work so well in these landscapes while sometimes remaining sensitive to apparently small changes.
Another subtlety is that different parameter configurations do not necessarily correspond to different learned functions.
Consider a two-neuron network:
If we exchange the two neurons, swapping both their input and output weights, the resulting function remains unchanged.
This is an example of permutation symmetry: two distinct parameter configurations represent exactly the same function.
More generally, different training runs can produce different weights that represent identical functions, similar functions, or genuinely different functions that happen to achieve comparable validation losses.
Therefore, inspecting final parameter values alone cannot establish whether two networks have learned meaningfully different behavior.
Their predictions and internal representations must also be examined.
Consider two classifiers that each achieve 95% accuracy on a test set of 100 examples. If both classifiers make mistakes on the same five examples, combining them may provide little benefit.
But if their errors occur on completely different examples, at least one classifier is correct on every example. A sufficiently reliable mechanism for selecting the correct prediction could potentially achieve perfect accuracy on that test set.
Simply averaging the predictions does not guarantee this improvement, but the example reveals an important principle:
Accuracy measures how frequently a model is correct. Error correlation helps us understand whether different models fail together.
This reinforces the broader lesson: neither final loss, overall accuracy, nor parameter values alone provides a complete description of what a network has learned.
We can summarize scientific understanding of deep learning at three levels. At the empirical level, we ask whether a technique works. At the mechanistic level, we ask how it changes the network. At the predictive level, we ask when it should work, why it should work, and under what conditions its advantage should disappear.
We can often answer the first question, sometimes answer the second, and still struggle with the third.
This is one reason deep learning has occasionally been compared to alchemy. Practitioners can combine architectural components, optimization algorithms, and training procedures to produce highly effective systems without possessing a complete theory explaining every interaction.
The analogy should not be taken to mean that deep learning lacks mathematical foundations. Rather, it highlights the gap between our ability to construct successful systems and our ability to predict their behavior from first principles.
Building a Predictive Theory Through Pre-training
If our objective is to develop reliable intuition, we should treat neural network training as a process whose outcomes depend on several interacting variables.
We can write this abstractly as
where represents architecture, the optimizer and training configuration, the data, the loss function, and the randomness introduced by initialization, data ordering, and other stochastic processes.
The function represents the training process, while represents its observable outcomes.
These outcomes include external performance measures, such as training loss, validation loss, convergence speed, and downstream accuracy. They also include internal properties, such as activation magnitudes, gradient norms, parameter updates, and signal propagation through the network.
Two models may achieve similar final losses while exhibiting significantly different internal training dynamics. Understanding both aspects is necessary if we want to predict the consequences of changing a model's design.
Language-model pre-training provides a particularly useful setting for developing this understanding.
The basic objective of pre-training is to approximate the probability distribution underlying a dataset, such as a large collection of internet text.
Suppose a language model encounters the context:
The capital of France is ___
Different tokens have different probabilities of occurring next. The model learns to assign probabilities to possible continuations based on the statistical relationships present in the training data.
Denote the underlying data distribution by and the learned model distribution by . Pre-training seeks parameters that make approximate under the chosen training objective.
This is more subtle than simply memorizing text. To perform well on unfamiliar sequences, a model must learn statistical relationships that generalize beyond individual training examples.
For an autoregressive language model, the probability assigned to a token sequence is factorized as
Here, denotes the tokens preceding position .
For example, the probability of the sequence The cat sleeps is
This factorization follows directly from the probability chain rule. It is a general property of joint probability distributions, rather than an approximation introduced by Transformers.
The model is trained by minimizing the expected negative log-likelihood of data sampled from :
If the model assigns a high probability to the observed sequence, its negative log-likelihood is small. If the model assigns a low probability, the loss is large.
Taking logarithms also converts the product of conditional probabilities into a sum:
Because longer sequences naturally accumulate more total loss, it is often useful to express the objective as the average negative log-likelihood per token:
This gives us a relatively simple and measurable objective for studying language-model training.
Pre-training is also attractive for empirical investigation because validation loss tends to provide a useful signal about downstream model quality. Scaling laws allow us to study relationships between model size, data, compute, and loss at smaller scales, then use those observations to make predictions about larger training runs.
These predictions are not exact guarantees, but they provide a framework for reasoning about experiments before spending the resources required to perform them at full scale.
Furthermore, training models from scratch offers a comparatively controlled setting: each run begins from an initialization rather than inheriting an already trained model whose previous optimization history may complicate interpretation.
Together, these properties make pre-training a useful environment for building and testing predictive theories of neural network behavior.
Yet an important question remains. If the training objective is fundamentally about assigning probabilities to text, why do sufficiently capable language models develop useful abilities in reasoning, translation, coding, and mathematical problem-solving?
The objective itself is simple to write down. The capabilities that emerge from optimizing it across large networks and datasets are much harder to predict.
This brings us back to the original question of why deep learning works.
We understand the mathematical operations used to construct neural networks. We know how gradients are calculated, how parameters are updated, and how training objectives are formulated. We also possess many empirical techniques that reliably improve model performance.
But knowing these ingredients does not mean that we can predict the behavior of every architecture produced by combining them.
A stronger understanding requires us to identify the conditions under which particular mechanisms matter, distinguish causal effects from correlations, and develop mental models that survive experimental testing.
The objective is not to avoid experimenting until we have a complete theory. Experiments are often what reveal the mechanisms that our theories are missing.
Instead, we should continuously move between intuition and evidence: develop an explanation, turn it into a prediction, design an experiment that could contradict it, and revise the explanation based on what happens.
Ultimately, deep learning expertise is not merely knowing which techniques work. It is understanding why they work, predicting when they will work, recognizing the limits of those predictions, and improving those predictions through experimentation.