Model complexity

One of the most consequential choices when training a predictive model is how complex to make it. A model with few parameters is easy to interpret and cheap to estimate, but may miss genuine non-linear patterns in the data. A model with many parameters — potentially far more than observations — can in principle represent any relationship, but risks memorising noise rather than signal. The classical answer, encoded in the bias-variance tradeoff (see Section Validating and tuning), is to find a middle ground where both sources of error are balanced.

Over the past decade, this textbook answer has been substantially revised. Deep neural networks and other modern learners routinely operate in a regime where the number of parameters exceeds the number of training observations by orders of magnitude, yet generalise remarkably well. Far from being pathological, this overparameterised regime exhibits a novel phenomenon: as model capacity grows past the interpolation threshold — the point where training loss reaches zero — test error initially spikes but then descends a second time toward competitive or even superior performance. This empirical pattern, first systematically documented by Belkin et al. (2019), is now known as the double-descent curve and has been studied theoretically by Bartlett et al. (2020), Belkin et al. (2020), and Hastie et al. (2022), among others.

The finance literature has developed a specific instantiation of this idea. Kelly et al. (2024) establish, both theoretically and empirically, that ridgeless regression applied to a large number of features constructed via Random Fourier Features (RFF; Rahimi & Recht (2007)) achieves out-of-sample return-predictive performance that increases monotonically with feature dimension. They call this the virtue of complexity. A natural challenge comes from Nagel (2025), who argues that the empirical gains are largely an artefact of RFF implicitly constructing volatility-timed momentum signals on short rolling windows, rather than learning genuinely complex patterns from the cross-section of firm characteristics.

This chapter develops and tests these ideas directly. Section Section 11.1 briefly recalls the classical tradeoff. Section Section 11.2 introduces RFF as a formal complexity dial. Section Section 11.3 is the empirical core: we implement the double-descent experiment on our equity dataset, showing the catastrophic spike at the interpolation threshold and the subsequent recovery under ridgeless regression, alongside the well-behaved, monotone curve of ridge regression. Sections Section 11.4 and Section 11.5 survey the recent finance literature, from the scaling-laws perspective of Timmermann & Vulicevic (2026) to the agent-based implications of Molavi et al. (2024) and the theoretical underpinnings of Fallahgoul (2025).

11.1The classical view and its limits¶

The classical bias-variance tradeoff (discussed in Section Validating and tuning) tells us that the expected test error of a model decomposes as:

As model complexity grows — for instance as the number of features $D$ increases — bias decreases (the model can fit richer functions) but variance increases (small changes in training data produce large changes in the estimated function). The optimal model minimises their sum. Translated into the language of penalised regression, this is precisely what ridge regression does: the penalty $\lambda \|\boldsymbol{\beta}\|^2$ shrinks coefficients, trading off a modest bias for a substantial variance reduction.

This picture is coherent and correct — within the underparameterised regime where $D < n$. It breaks down as soon as $D$ approaches $n$: at the interpolation threshold $D \approx n$, the feature matrix becomes square or nearly square, the minimum-norm least-squares estimator (ridgeless regression) attempts to perfectly fit all training observations, and the variance explodes. Classical theory predicts that test performance should be catastrophically bad for all $D \geq n$.

What is observed in practice is more nuanced. For $D > n$, the overparameterised model has infinitely many solutions that interpolate the training data; it picks the one with minimum $L^2$ norm. Bartlett et al. (2020) show that, under sufficient effective regularisation from the data geometry (many near-zero eigenvalues of the feature covariance), this minimum-norm interpolator can generalise well. The implicit regularisation of the minimum-norm solution plays the role of explicit penalisation in ridge regression, but only when the spectrum of the feature matrix has the right structure. Hastie et al. (2022) characterise the precise spectral conditions under which benign overfitting occurs, and derive closed-form limits for the test error as a function of the overparameterisation ratio $D/n$.

11.2Random Fourier Features as a complexity dial¶

Random Fourier Features (RFF; Rahimi & Recht (2007)) provide an elegant mechanism to continuously vary model complexity while preserving a clear functional form. The key insight is Bochner’s theorem: any continuous, shift-invariant kernel $k(\mathbf{x}, \mathbf{x}')$ can be written as the Fourier transform of a non-negative spectral density $p(\boldsymbol{\omega})$. For the Gaussian (RBF) kernel $k(\mathbf{x}, \mathbf{x}') = \exp(-\gamma\|\mathbf{x} - \mathbf{x}'\|^2)$, the spectral density is itself a Gaussian.

The RFF approximation constructs an explicit $D$-dimensional feature map $\boldsymbol{\phi}: \mathbb{R}^K \to \mathbb{R}^D$ such that:

where $\boldsymbol{\omega}_j \overset{\text{iid}}{\sim} \mathcal{N}(\mathbf{0}, 2\gamma\mathbf{I}_K)$ and $b_j \overset{\text{iid}}{\sim} \mathrm{Uniform}(0, 2\pi)$ are drawn once and fixed. A linear model trained on $\boldsymbol{\phi}(\mathbf{x})$ then implicitly implements a kernel machine, and the quality of the approximation improves as $D \to \infty$.

In our context, starting from $K = 7$ firm characteristics, we can map any observation to a feature vector of dimension $D \gg K$. By varying $D$ from the underparameterised regime ($D < n$) through the interpolation threshold ($D \approx n$) and deep into the overparameterised regime ($D \gg n$), we obtain a clean one-parameter experiment that traces the full double-descent curve. This is precisely the design used by Kelly et al. (2024), who report that ridgeless regression on RFF features achieves monotonically improving test performance for $D \gg n$ — the virtue of complexity.

Fallahgoul (2025) provides a theoretical perspective specific to financial data. He proves that within-sample feature standardisation — a common pre-processing step — fundamentally alters the approximation properties of RFF, and derives information-theoretic lower bounds showing that reliable recovery of the true regression function requires sample sizes far exceeding those typically available in finance. The implication is that apparent gains from RFF complexity may not arise from genuine function recovery, but from a different mechanism that Fallahgoul (2025) calls projection learning: the model learns increasingly accurate projections of the true function onto the observable feature span, a weaker but still useful goal.

11.3The double-descent in equity return prediction¶

11.3.1Setup¶

We implement the RFF complexity experiment on our equity dataset. To make the interpolation threshold visible at a computationally tractable feature count, we subsample $n = 500$ training observations from the training set (all observations before 2017). The test set consists of all available observations from 2017 onward. Both sets use the same seven standardised characteristics.

We vary the number of random features $D$ from 7 (the raw feature dimension) up to 4,000 (an 8-fold overparameterisation). For each $D$ we fit two models:

• Ridgeless regression (alpha$= 10^{-9}$): minimum-norm interpolation.

• Ridge regression (alpha$= 1$): classical penalised regression.

11.3.2The double-descent curve¶

Figure 11.1 makes the double-descent crystal clear on real equity data. Three phases are visible:

1. Classical regime ($D < n$). Both ridgeless and ridge models improve as $D$ grows, since additional features reduce approximation bias. Their test MSE values are close: ridge’s explicit penalty and ridgeless’s effective implicit penalty from the low-rank RFF matrix produce similar solutions.

2. Interpolation threshold ($D \approx n = 500$). At this point, ridgeless regression attempts to exactly interpolate all 500 training observations with the full capacity of the feature map. The minimum-norm solution is valid but is dominated by noise, causing test MSE to spike catastrophically (the actual peak exceeds 1,200× the return variance; it is clipped here). Ridge regression, insulated by its penalty, exhibits no such pathology.

3. Overparameterised regime ($D > n$). With more features than observations, there are infinitely many exact interpolators; ridgeless selects the minimum $L^2$-norm one, which in the high-dimensional limit acts as an implicit regulariser. Test MSE decreases — this is the second descent. Ridge remains flat and stable, reflecting the fact that explicit regularisation already prevents the variance explosion.

A key quantitative message: even at $D = 4{,}000$ (8× overparameterisation), ridgeless regression has not recovered to the performance level of a well-tuned ridge regression. This is characteristic of financial return data: the signal-to-noise ratio is very low, so the implicit regularisation of minimum-norm interpolation is insufficient to counter the noise amplification in the overparameterised regime. Ridge’s explicit penalty dominates.

11.3.3OOS $R^2$ as a function of complexity¶

Figure 11.2 translates the MSE picture into the OOS $R^2$ metric familiar from Chapter Predictability of asset returns. Ridge regression delivers a modest but positive or near-zero OOS $R^2$ across the full range of $D$ values, reflecting the benefit of additional non-linear features even in the well-regularised case. Ridgeless regression suffers catastrophically near the interpolation threshold (clipped in the figure), and its recovery in the overparameterised regime does not restore the level of ridge.

This pattern is the direct financial counterpart of Figure 11.1. From a practitioner’s standpoint:

• Adding features via RFF does help, but only with proper regularisation. Ridge with a moderate $\lambda$ captures the non-linearity introduced by the random feature map without the variance explosion.

• Ridgeless regression in the overparameterised regime does not consistently beat the naive mean in our data, suggesting that — at least on these characteristics over this sample — the conditions for benign overfitting are not met.

• The optimal $D$ for ridge is roughly in the range where the Gaussian kernel is well approximated (typically $D \sim 200$–500). Beyond this, there are diminishing returns.

11.3.4Sensitivity to the training window size¶

The double-descent spike occurs at $D \approx n$, so its location shifts with the training sample size. A natural robustness check is to vary $n_{\text{sub}}$ and confirm that the interpolation threshold tracks with it.

Figure 11.3 confirms the theoretical prediction: the interpolation threshold — the location of the double-descent spike — occurs at $D = n$, regardless of how many observations are available. Doubling the sample size from 500 to 1,000 shifts the peak to the right by one step on the log scale; halving it to 250 shifts it to the left. This is not a peculiarity of financial data but a universal property of minimum-norm interpolation. It also explains why the virtue-of-complexity phenomenon is most relevant in settings with large training sets: when $n$ is large, reaching the overparameterised regime (and benefiting from the second descent) requires a correspondingly larger feature expansion.

11.4The virtue of complexity: finance debate¶

11.4.1The original claim¶

Kelly et al. (2024) establish that ridgeless regression applied to RFF-expanded features achieves monotonically improving OOS return prediction as $D \to \infty$, provided the model is estimated on a rolling window of fixed length (rather than an expanding window). The key design choices are:

1. A fixed rolling training window (e.g., 240 months), so the sample size $n$ stays constant as the model is re-estimated each period.

2. No explicit regularisation: ridgeless regression, which converges to a kernel machine as $D \to \infty$.

3. A sufficiently large $D$ to enter the virtuous overparameterised regime.

The theoretical result of Kelly et al. (2024) is clean: the limiting ($D \to \infty$) ridgeless estimator equals the kernel ridge regression (KRR) with a Gaussian kernel, and adding more features continuously interpolates between OLS and KRR. Because KRR is well-regularised by its kernel structure, the limiting model generalises well even without an explicit penalty.

In our experiment above we use an expanding window (rather than a rolling window), which is why we do not observe the same monotone improvement. With a fixed rolling window, the relationship between $n$ and $D$ changes in a controlled way: as $D$ grows, the degree of overparameterisation $D/n$ grows, and — according to the theory — so does test performance. The fixed window also ensures stationarity of the model across time, which is a non-trivial advantage in a non-stationary setting.

Timmermann & Vulicevic (2026) provide a macro-level perspective. They fit power-law scaling relationships between return forecast accuracy and the scale of computation (measured by model parameters, training data, and compute budget). Their empirical finding is striking: over the past 30 years, improvements in compute account for more than 80% of the documented gains in return predictability. Larger models trained on more data have genuinely improved the efficiency of financial markets. Quantitatively, a 25% increase in compute raises a long-short equity strategy’s Sharpe ratio by approximately 10% in their sample.

11.4.2The critique¶

Nagel (2025) offers a sharp reinterpretation. He demonstrates analytically that, under a rolling-window estimation scheme, the RFF-based ridgeless predictor converges to a weighted average of past returns as $D$ grows — that is, to a form of volatility-adjusted momentum. The mechanism is as follows. In the overparameterised regime, the minimum-norm solution gives high weight to the training observations that are most easily “reached” in the high-dimensional feature space — and those correspond to recent observations with large returns. The resulting strategy looks like a complex, non-linear learner but is mechanically equivalent to a momentum signal.

The empirical implication is that the strong OOS performance of the virtue-of-complexity model in the Kelly et al. (2024) sample is not evidence of the model having learnt complex non-linear relationships, but rather coincidental historical success of the momentum factor during the relevant period. When Nagel recasts the comparison against a properly benchmarked momentum strategy, the gains largely vanish.

Kılıç (2025) extend the investigation to exchange rate prediction, a different asset class where momentum is less dominant. They find that RFF complexity delivers at most localised, fragile gains in small samples — and that as the training window expands, linear regression often matches or exceeds the complex model. This corroborates Nagel’s general point: the virtue of complexity may be a special case that depends critically on the asset class, the sample period, and the rolling window length.

11.4.3Global versus regional models¶

Chen et al. (2025) investigate a distinct dimension of complexity: pooling training data across regions versus estimating region-specific models. Using data from 24 developed markets, they find that linear models benefit from regional parsimony (local models do better), whereas more complex ML algorithms — with their ability to exploit non-linearities — benefit from the richer global dataset. This suggests a two-way interaction: complexity is most virtuous when the training set is large. A global dataset provides enough observations to make the overparameterised regime relevant. This is fully consistent with the scaling-law perspective of Timmermann & Vulicevic (2026): larger data and more compute reinforce each other.

11.4.4Nonlinearity and distributional structure¶

Huang (2025) takes a theoretical approach to ask when nonlinear models are most valuable. Using an equilibrium model, he shows that non-linear machine learners dominate linear ones when stock payoffs deviate from elliptical distributions — specifically, when idiosyncratic skewness is high and growth-related firm characteristics are prominent. This connects the virtue of complexity to the distributional properties of the cross-section rather than solely to the overparameterisation ratio. The practical implication is that complex models should be deployed selectively: they add the most value for growth stocks, especially during periods of elevated market volatility, and less value for large-cap value stocks with near-Gaussian payoffs.

11.5Theoretical perspectives¶

11.5.1Projection learning¶

The most puzzling aspect of the virtue-of-complexity literature is that its strong results appear to hold even when the number of parameters vastly exceeds the number of observations — a regime where classical theory predicts failure. Fallahgoul (2025) provides a reconciliation through the concept of projection learning.

The core insight is that a model does not need to recover the true regression function $f^*$ — it only needs to find the best projection of $f^*$ onto the span of the feature map $\boldsymbol{\phi}$. In a high-dimensional RFF space, this span is rich, and even a noise-dominated estimator may project onto it with useful accuracy. Formally, let $\hat{f}_D$ be the ridgeless estimator with $D$ features. As $D \to \infty$, $\hat{f}_D$ converges to the best $L^2$ projection of $f^*$ onto the reproducing kernel Hilbert space (RKHS) of the Gaussian kernel, which is a meaningful object even when estimation error is high. The key formula derived by Fallahgoul (2025) shows that the projection quality improves monotonically in $D$ regardless of the level of estimation error — explaining both the virtue of complexity and its limit.

Fallahgoul (2025) complements this with information-theoretic bounds. He proves that reliably recovering $f^*$ in typical financial settings requires sample sizes far beyond what is available — often several orders of magnitude larger. The implication is that the literature should not interpret the empirical gains from complexity as evidence of function recovery, but rather as projection learning operating within a specific kernel RKHS. Fallahgoul (2025) offers a modular, non-random-matrix-theory alternative that decomposes prediction error into monotonically decreasing approximation bias and non-monotonic estimation variance, providing a simpler diagnostic toolkit.

11.5.2Large and deep factor models¶

Kelly et al. (2026) push the complexity frontier to deep neural networks for stochastic discount factor (SDF) estimation. They introduce the Portfolio Tangent Kernel (PTK) — a kernel derived from the SDF learned by a deep network — that enables a clean additive decomposition of the SDF into a feature-learning component and a pricing component. Their key empirical finding is that the spectral complexity of learned factors — measured by the effective number of informative directions in the factor space — has increased six-fold since the early 2000s. This coincides with the well-documented performance deterioration of simple anomaly strategies (see Chapter Factor investing and asset pricing anomalies), and is interpreted as evidence of increasing market efficiency: as more capital exploits the same anomalies, the marginal information value of each factor dimension shrinks, pushing the frontier of profitable prediction into increasingly complex, high-dimensional territory.

11.5.3Complexity in agents’ minds¶

Molavi et al. (2024) approach complexity from a different angle: not the complexity of the estimator used by an econometrician, but the complexity of the model in agents’ minds. When the true data-generating process is more complex than investors can represent, they approximate it with a simpler model — essentially fitting a lower-dimensional projection. This rational inattention to complexity generates systematic forecast errors and, crucially, asset pricing anomalies. The forward discount puzzle and predictability reversal patterns observed in fixed income and currency markets are shown to be consistent with agents using model complexities that are too low relative to the true process. This is the demand-side mirror of the supply-side question: practitioners who use simple models leave money on the table; agents who use simple models in equilibrium create the patterns that sophisticated models exploit.

11.6Conclusion¶

The debate around model complexity in asset pricing is one of the most active in contemporary empirical finance. The key messages from this chapter are:

• Double descent is real and clearly visible in equity return data (Figure 11.1). At the interpolation threshold $D = n$, ridgeless regression fails catastrophically; above it, performance partially recovers — but not to the level of a well-tuned ridge regression, because financial return noise is too high for benign overfitting.

• RFF provides a clean complexity dial. By varying the number of random features $D$, one continuously transitions from linear regression (small $D$) to Gaussian kernel regression (large $D$), with the interpolation threshold at $D = n$ providing the critical transition point. The threshold shifts exactly with $n$ (Figure 11.3).

• Regularisation is essential. Ridge regression dominates ridgeless in the financial setting considered here. The virtue of complexity claimed by Kelly et al. (2024) relies on specific design choices (fixed rolling window, very large $D$) under which the implicit regularisation of ridgeless regression is sufficient. Nagel (2025) shows this may reduce to a momentum effect in practice.

• Theory provides guardrails. Projection learning (Fallahgoul (2025)), information-theoretic bounds (Fallahgoul (2025)), scaling laws (Timmermann & Vulicevic (2026)), and agent-based complexity (Molavi et al. (2024)) together offer a coherent picture of when complexity helps, why it helps, and what its limits are.

• Context matters. Complexity is most valuable when the training set is large (Chen et al. (2025)), when the return distribution is non-elliptical (Huang (2025)), and when the asset class is amenable to complex patterns — none of which is uniformly true across markets and periods.

11.7Exercises¶

1. Varying gamma: The bandwidth parameter gamma in the RBF kernel controls how quickly the kernel decays with distance. Re-run the double-descent experiment with gamma $\in \{0.1, 0.3, 1.0, 3.0\}$. Does the location or height of the interpolation spike change? Does the post-threshold recovery improve with larger gamma?

2. Ridge tuning: Instead of a fixed alpha = 1.0, use cross-validation (e.g., RidgeCV) to select the optimal regularisation for each value of $D$. Does this change the shape of the ridge OOS $R^2$ curve in Figure 11.2?

3. Rolling window: Following the design of Kelly et al. (2024), replace the subsample of $n$ observations with a rolling window of fixed length (e.g., the most recent 36 months). For each test date, re-estimate the model on the rolling window and compute the OOS $R^2$. Compare the ridgeless and ridge performance under this protocol to the expanding-window results from Chapter Predictability of asset returns.

4. Kernel comparison: RFF with a Gaussian kernel implicitly estimates a smooth, infinitely differentiable function. Repeat the experiment using an approximation of the Laplace kernel (where $\boldsymbol{\omega}_j$ are drawn from a Cauchy distribution). Which kernel produces a better OOS $R^2$ on the financial data? What does this imply about the smoothness of the true return-prediction function?

5. The momentum connection: Inspired by Nagel (2025), compute the Spearman rank correlation between (a) the predictions of the ridgeless RFF model with $D = 2{,}000$ and (b) the raw Mom_LT momentum signal. Does this correlation increase as $D$ grows? What does it suggest about the nature of the complex model’s predictions?