AI Models of Loss Aversion and Hyperbolic Discounting

Behavioral economists increasingly rely on artificial intelligence to understand why human choices deviate from traditional economic rationality. This article provides an overview of how researchers combine hybrid cognitive architectures and reinforcement learning (RL) to accurately simulate prospect-theory loss aversion and hyperbolic discounting. By merging rule-based psychological frameworks with data-driven machine learning algorithms, scientists can recreate the exact neural and cognitive processes that cause humans to fear losses more than they value gains, and to impulsively favor immediate rewards over larger future payouts.

Traditional reinforcement learning agents are designed to maximize expected utility, making them far too rational to accurately represent human behavior. To correct this, behavioral economists alter the fundamental mathematical structures of these AI systems. They replace standard utility functions with prospect-theory value functions and swap out consistent time-preferences for unstable, human-like discounting formulas. Furthermore, they embed these modified RL algorithms into hybrid cognitive architectures, which mimic the dual-process nature of the human brain.

Modeling Loss Aversion via Asymmetric Reinforcement Learning

In prospect theory, loss aversion dictates that the psychological pain of losing is roughly twice as powerful as the pleasure of gaining an equivalent amount. Standard RL algorithms update their decision-making policies using a Reward Prediction Error (RPE), which treats positive and negative outcomes symmetrically.

To model loss aversion, economists introduce asymmetric learning rates into the RL agent's RPE calculations. This is achieved through two primary mechanisms:

  • Reference Point Integration: Instead of evaluating absolute wealth, the AI agent is programmed to evaluate outcomes relative to a dynamic reference point, such as its previous state or an expected baseline.
  • Asymmetric Updating: The algorithm applies different weights to positive and negative prediction errors. When an outcome falls below the reference point (a loss), the negative prediction error is multiplied by a loss-aversion parameter. This forces the AI to over-correct its behavior to avoid future losses, successfully mimicking human risk-aversion in mixed-gamble scenarios.

Simulating Hyperbolic Discounting in Temporal Difference Learning

Standard RL models use exponential discounting, where the value of a future reward decreases at a constant rate over time. This results in time-consistent preferences. Humans, however, exhibit hyperbolic discounting: our valuation of a reward drops precipitously for short delay periods, then declines more gradually over longer horizons. This explains why a person might prefer one dollar today over two dollars tomorrow, yet prefer two dollars in 366 days over one dollar in 365 days.

Economists use Temporal Difference (TD) learning modifications to model this impulsivity:

  • Hyperbolic Discount Factors: Researchers replace the standard exponential discount factor (\(\gamma^t\)) in the Bellman equation with a hyperbolic function, typically \(1 / (1 + kt)\), where \(k\) represents the degree of impatience and \(t\) represents time.
  • Multi-Horizon Agents: Another approach involves ensembles of standard exponential RL agents operating at different time scales simultaneously. When their outputs are aggregated, the collective behavior naturally produces a hyperbolic discounting curve, simulating how different parts of the brain compete over short-term and long-term goals.

The Role of Hybrid Cognitive Architectures

While modified RL algorithms can replicate biased data, they do not fully explain the cognitive constraints—like memory limits and attention spans—that drive these biases. To bridge this gap, behavioral economists embed RL algorithms into hybrid cognitive architectures such as ACT-R (Adaptive Control of Thought-Rational) or Clarion.

These architectures explicitly model the human brain's "System 1" (fast, automatic, unconscious) and "System 2" (slow, deliberative, conscious) processes:

  • The Sub-Symbolic Layer (System 1): This layer utilizes the modified reinforcement learning algorithms described above. It rapidly processes environmental cues and generates gut-level valuations skewed by loss aversion and immediate reward bias.
  • The Symbolic Layer (System 2): This layer uses explicit, rule-based logic and working memory. It attempts to calculate optimal long-term strategies, overriding the impulsive sub-symbolic layer when cognitive resources are high.

By adjusting the interaction between these two layers, economists can simulate how stress, cognitive load, or time pressure degrades System 2's oversight. When the symbolic layer is overwhelmed, the hybrid agent defaults to its RL-driven sub-symbolic layer. Consequently, the AI exhibits heightened loss aversion and steep hyperbolic discounting, providing economists with a precise, mechanistic model of human irrationality under real-world constraints.