What does your shopping reveal? A math model reads between the lines of the receipt

Imagine walking into a supermarket. You buy pasta, milk, maybe that tempting ice cream. Now imagine doing this every week for years. You create a mountain of data: what you buy, when, how much you spend, if you switch brands. For researchers, this is "high-dimensional multivariate longitudinal data": fancy words meaning many variables (products) measured many times (weeks).

Making sense of it is a statistical nightmare. Variables are linked (pasta buyers often buy sauce), change over time (summer vs winter diet), and sometimes go missing (vacation week).

A new study on arXiv proposes an elegant solution: **latent variables**. These are "hidden" factors we don't see directly — like "urge to cook", "weekly budget", "health consciousness" — but which link all visible purchases together.

The model handles mixed data (quantities, yes/no, categories), time gaps, and finds these invisible forces. It then uses a central limit theorem to give mathematical certainty to predictions.

Tested on real grocery receipts, the system learns to predict future carts and explain *why* we buy certain things together. It's not magic, it's linear algebra spotting human patterns.

Next time you scan your loyalty card, remember: behind that "beep" there's math trying to understand your hidden desires.

The paper introduces a latent factor model for high-dimensional multivariate longitudinal data with mixed-type outcomes (Gaussian, Binomial, Poisson). The covariance structure is decomposed into a low-rank component (latent factors) capturing cross-variable and cross-time dependence, and an idiosyncratic component. This addresses the "curse of dimensionality" when the number of variables $p$ exceeds the number of subjects $n$ or time points $T$.

For inference, the authors develop an EM-based or variational inference estimator, establishing a Central Limit Theorem for regression coefficients $\beta$ under regularity conditions allowing $p, T \to \infty$. This enables valid hypothesis testing and confidence intervals in high-dimensional settings.

The number of factors $r$ is selected via a BIC-type information criterion adapted for longitudinal structure, penalizing model complexity consistently. The model natively handles non-monotone missing data (e.g., right-censoring, drop-out) assuming Missing At Random (MAR), leveraging the estimated covariance structure for implicit imputation during the E-step. Application to real grocery shopping data shows latent factors correspond to interpretable consumption profiles (e.g., "weekly staple shop", "fresh produce", "snacks\)), outperforming baselines like VAR or standard mixed-effects models in predictive accuracy (MSE) and imputation capability.