How can we merge the uncertainty of multiple models into a single prediction?

Imagine having several expert friends giving you advice on the same decision: each is skilled, but their opinions differ. In artificial intelligence the same happens with predictive models. When multiple algorithms trained on the same task produce different results, it’s essential to gauge how confident they are. This is where conformal prediction (CP) steps in, a technique that builds prediction sets with a statistical coverage guarantee.

The new method SACP – Symmetric Aggregated Conformal Prediction – suggests gathering the non‑conformity scores (values indicating how far a prediction deviates from the data) from all models, turning them into e‑values (a kind of inverted probability) and merging them with a symmetric aggregation function, such as the mean or median. This flexibility lets you pick the best strategy for the dataset, yielding tighter yet still reliable prediction sets.

The authors tested SACP on various datasets, both regression and classification, and observed a consistent boost in efficiency compared to traditional aggregation methods. In practice, SACP makes predictions more precise without sacrificing statistical safety, opening doors for applications where uncertainty matters, like medicine, finance, or robotics.

SACP rests on two key concepts: non‑conformity scores (NCS) and e‑values. NCS, typically computed as the distance between the observed response and the model's predicted distribution, quantify how much a single observation deviates from the model. In conformal prediction, these scores are ordered to construct a coverage interval at level 1‑α.

Transforming them into e‑values draws from the Bayesian evidence framework, where an e‑value is a non‑negative random variable with expected value ≤ 1 under the null hypothesis. Converting NCS into e‑values enables the exploitation of additive properties and combination via symmetric functions, ensuring that CP validity is preserved after aggregation.

Choosing the aggregation function is crucial. Functions such as the arithmetic mean, median, or geometric mean offer different trade‑offs between robustness to outliers and sensitivity to model variations. Symmetry guarantees that the order of models does not affect the final result, making the method suitable for scenarios with a variable number of predictors.

Theoretically, the authors prove that under exchangeability of scores, symmetric aggregation maintains the desired marginal coverage. They also provide bounds on the size of the predicted set, showing that SACP can yield tighter intervals compared to union or intersection‑based methods.

Empirically, SACP was evaluated on benchmarks like UCI, MNIST, and CIFAR‑10, using base models including Random Forest, Gradient Boosting, and deep neural networks. Results indicate an average reduction of 10‑15% in prediction set length while preserving the required 95% coverage. This makes SACP especially appealing for high‑reliability applications where every uncertainty reduction translates into more informed decisions.