The Unreasonable Difficulty of Time Series Forecasting

Published: June 27, 2026

Recently, I have been contemplating the inherent complexities that make time series forecasting uniquely challenging when compared to standard IID (Independent and Identically Distributed) Machine Learning problems or other sequence-based learning tasks.

📊 Benchmarking the Struggle

To illustrate this difficulty, I conducted a series of experiments comparing various modeling paradigms against several benchmark datasets: m4_hourly, m4_daily, etth, exchange, electricity, and bitcoin.

Model Categories Tested

Note: For visual clarity, historical data was truncated to three times the length of the forecast window.

Key Findings

The Main Takeaway: For datasets exhibiting stable and strong seasonality (e.g., m4_hourly), the zero-shot foundation models and simple statistical baselines (like MSTL and Seasonal-Naive) outperform the rest by a significant margin.

Conversely, highly sophisticated models often fail spectacularly. They frequently trend in the entirely wrong direction, making them superior no better than a basic naive predictor.

One might assume this is because models lack causal information (e.g., geopolitical events affecting exchange rates). However:

1. Even with exogenous features, traders aren't simply "printing money."
2. In many other ML domains, we lack full causal data yet still achieve results far better than a random walk.

This suggests the problem is deeper than just "missing features."

---

🔍 Why is Forecasting So Hard?

1. Defining the Problem

At its core, forecasting is the attempt to estimate future values of a random variable based on a historical sequence $y_{1:t}$. We seek a function $f$ that predicts the value at a future horizon $h \ge 1$:

$\hat{y}_{t+h} = f(y_{1:t})$

Variations of the task include:

• Exogenous Forecasting: Incorporating external drivers $x$ (e.g., price changes, weather): $f(y_{1:t}, x_{1:t})$.
• Probabilistic Forecasting: Predicting a distribution rather than a single point: $p(y_{t+h} \mid y_{1:t})$.

2. DGP vs. DGD: The Fundamental Divide

Modern Deep Learning is designed to learn a non-linear predictor $f_\theta: X \rightarrow Y$. In standard ML, we assume samples $(x, y)$ are drawn from a Data Generating Distribution $p(x, y)$, and we minimize the expected loss:

$\min_\theta \mathbb{E}_{(x, y)\sim p(x, y)} [\ell (f_\theta (x), y)] \approx \min_\theta \frac{1}{n}\sum_{i=1}^n [\ell (f_\theta (x_i), y_i)]$

However, time series data does not follow this IID logic. Instead, it emerges from a Data Generating Process (DGP). We aren't looking at $n$ independent samples, but rather a single trajectory through time.

The sequence is a joint probability distribution over the entire path: $p(y_1, y_2, \cdots, y_t) = p(y_1) \prod_{i=2}^t p(y_i | y_{i-1}, y_{i-2}, \cdots, y_1)$

3. The Core Challenges

Because the IID assumption is discarded, we encounter three primary hurdles:

• Low Signal-to-Noise Ratio: Time series are often "feature poor." While LLMs succeed in sequence prediction because tokens have high mutual information and semantic richness, time series signals are often drowned out by noise.
• The "Lack of Data" Paradox: We may have millions of timestamps, but high auto-correlation shrinks the effective sample size.
• Distribution Shift: Time series are hyper-susceptible to non-stationarity.

Understanding Effective Sample Size

If data is highly correlated, $T$ timestamps do not provide $T$ independent pieces of information. In the quest to predict $y_{t+1}$, the value of $y_t$ provides most of the information, leaving little new signal to learn.

The real unit of learning is the cycle, not the timestamp.

# Conceptual example of effective sample size
def calculate_effective_samples(total_timestamps, correlation_factor):
    # High correlation reduces the unique information available
    return total_timestamps / correlation_factor 

# 10,000 daily points with high correlation might only 
# provide the signal equivalent of 10 unique economic cycles.

For instance, if one is attempting to model the patterns preceding an economic recession, the "data" isn't the number of days in the dataset, but the $m$ number of previous recessions that have actually occurred.