Why Do Bots Raise Prices When Demand Booms?

Imagine an online store where prices aren’t set by a manager but by an algorithm that learns on its own. Researchers tested such algorithms using Q‑learning, an AI technique that learns by trial and error. They simulated a market where demand can suddenly jump or drop, i.e., experience demand "shocks." When the algorithms can see these changes, they adjust prices accordingly. If their discount factor (a parameter that measures how much they value the future) is high, the bots become pro‑cyclical: they raise prices during high‑demand periods, just like a seller who wants to cash in on the crowd. If the discount factor is low, the bots act counter‑cyclically: they lower prices when demand is high and raise them when it’s low, trying to maximize short‑term profit. In both cases, the algorithms manage to keep profits above those of a perfectly competitive market, showing that simple demand observability doesn’t stop algorithmic collusion. The surprising outcome is that, even without explicit agreements, bots can mimic price dynamics predicted by traditional economic theory simply by learning from demand fluctuations.

Q‑learning is a form of reinforcement learning where an agent learns an optimal policy by maximizing a Q‑value function Q(s, a), which estimates the expected return of taking action a in state s. In pricing contexts, the state can include current demand information, price history, and competitors' signals. The agent updates Q(s, a) using the Bellman equation: Q_{t+1}(s,a)= (1-\alpha) Q_t(s,a)+\alpha [r+\delta \max_{a'} Q_t(s',a')], where \alpha is the learning rate, r is the immediate profit, and \delta is the discount factor. A high \delta makes the agent more future‑oriented, favoring strategies that preserve a long‑term collusive reputation, typical of pro‑cyclical pricing. Conversely, a low \delta pushes the agent to maximize short‑term profit, leading to counter‑cyclical prices that exploit immediate market opportunities. Demand "shocks" introduce unpredictable variations in state s, but because algorithms directly observe demand, they can condition their pricing policy on this information. This yields a dynamic form of collusion: agents need not explicitly communicate, yet they converge to a price equilibrium that depends on the temporal structure of their profit objective. Simulations show that even without strict market rules, the combination of Q‑learning, demand observability, and the chosen discount factor suffices to reproduce the price patterns described by Rotemberg and Saloner (1986). Moreover, the ability to sustain supracompetitive profits suggests regulators must consider not only data transparency but also the learning characteristics of pricing algorithms.