Conformal Optimistic Prediction: Can We Forecast with Tighter Margins?

Conformal Prediction builds prediction sets that guarantee a predefined coverage probability without assuming any data distribution. In online settings, where data stream in real time and may shift abruptly, traditional methods often produce very wide intervals to avoid errors. The new Conformal Optimistic Prediction (COP) algorithm aims to improve this. COP watches non‑conformity scores (how much a new observation deviates from the model) and, by estimating their cumulative distribution function, dynamically adjusts interval width. When data exhibit a predictable pattern, intervals shrink, yet coverage stays guaranteed even if the estimate is off. The authors prove a joint bound on coverage and regret, confirming the method’s theoretical soundness. Moreover, COP retains distribution‑free finite‑sample coverage regardless of learning rate and converges when scores are i.i.d. Experiments show COP yields shorter intervals than competing baselines without sacrificing coverage. Practically, this means more precise forecasts for sensor monitoring, finance, or real‑time data streams, where speed and reliability matter.

COP's core lies in estimating the cumulative distribution function (CDF) of non‑conformity scores. Practically, one builds an empirical function F̂(t)=\frac{1}{n}\sum_{i=1}^{n}\mathbf{1}\{s_i\le t\}, where s_i are observed scores. This CDF allows computing, for a coverage level 1-α, the quantile q_{1-α}=\inf\{t: F̂(t)\ge 1-α\}. The prediction interval is then centered on the current model and has a radius proportional to q_{1-α}. COP's novelty is introducing an optimism factor: it blends the empirical CDF with a trend‑based forecast (e.g., moving average or ARIMA) to anticipate the future distribution shape. This reduces interval variance when the pattern is stable, while a minimum width safeguard ensures coverage even if the forecast errs. Theoretically, the authors prove that for any data sequence coverage is at least 1-α (coverage bound) and that regret—defined as the cumulative difference between COP's interval width and the optimal width knowing the true distribution—is bounded by O(\sqrt{n}). Under i.i.d. score assumptions, the empirical CDF converges almost surely to the true CDF, making COP intervals approach optimal ones. COP permits various learning rates η_t for updating the CDF, from constant to decreasing, without breaking coverage guarantees, suiting both rapid‑change (high η) and stable (low η) scenarios. Experiments on synthetic and real regression datasets (e.g., energy consumption forecasts) show a 10‑20% reduction in average interval length compared to Adaptive Online Conformal Prediction or Split Conformal, while keeping coverage within 1‑2% of the target.