How functools lru_cache Optimizes Python Functions

This article explains how Python's built-in functools.lru_cache decorator optimizes repetitive, computationally expensive function calls. You will learn the mechanics behind memoization, how the Least Recently Used (LRU) eviction strategy manages memory, how to implement the decorator with practical examples, and the key conditions required for using it effectively.

What is `functools.lru_cache`?

functools.lru_cache is a decorator in Python's standard library designed to implement memoization. Memoization is an optimization technique where the return values of expensive function calls are stored in memory. When the function is subsequently called with the exact same arguments, the cached result is returned immediately, bypassing redundant execution.

The decorator utilizes an LRU (Least Recently Used) cache algorithm, which ensures that the cache does not grow indefinitely. Once the cache reaches its defined size limit, it automatically discards the items that have not been accessed for the longest period.

How It Works Internally

Argument Hashing: When a decorated function is invoked, lru_cache evaluates the positional and keyword arguments passed to it. It combines these arguments into a hashable key.
Lookup: The decorator checks its internal dictionary to see if the key already exists:
- Cache Hit: If the key is present, the stored result is returned instantly. The entry is marked as recently used.
- Cache Miss: If the key is absent, the function executes normally. The resulting value is stored in the cache alongside the key.
Eviction: If adding a new entry causes the cache to exceed its configured maxsize, the least recently accessed key-value pair is evicted.

Because inputs are stored as dictionary keys, all arguments passed to a cached function must be hashable (e.g., integers, strings, tuples). Passing unhashable types like list or dict raises a TypeError.

Performance Comparison: A Practical Example

The impact of lru_cache is most evident in recursive algorithms, such as computing Fibonacci numbers.

Without Caching:

def fibonacci(n):
    if n < 2:
        return n
    return fibonacci(n - 1) + fibonacci(n - 2)

# Computing fibonacci(35) takes several seconds due to O(2^n) time complexity.
fibonacci(35)

In this unoptimized version, the function repeatedly recalculates identical subproblems (e.g., fibonacci(3) is computed millions of times).

With `lru_cache`:

from functools import lru_cache

@lru_cache(maxsize=128)
def fibonacci(n):
    if n < 2:
        return n
    return fibonacci(n - 1) + fibonacci(n - 2)

# Computing fibonacci(35) executes in fractions of a millisecond (O(n) time complexity).
print(fibonacci(35))

By caching previous results, the time complexity drops from exponential to linear.

Key Configuration Parameters

The @lru_cache decorator accepts two optional arguments:

maxsize (default: 128): Sets the maximum number of entries the cache can hold.
- Setting maxsize=None disables the LRU eviction logic, allowing the cache to grow without bound.
- In Python 3.9+, you can use @functools.cache as a shorthand for @lru_cache(maxsize=None).
typed (default: False): When set to True, arguments of different data types are cached separately. For example, f(3) and f(3.0) will be treated as distinct calls.

Monitoring and Managing Cache Performance

Functions wrapped with lru_cache provide built-in methods for inspection and maintenance:

cache_info(): Returns a named tuple showing cache metrics:
```
info = fibonacci.cache_info()
print(info)
# Output: CacheInfo(hits=33, misses=36, maxsize=128, currsize=36)
```
- hits: Number of times a result was retrieved from cache.
- misses: Number of times the function had to compute a result.
- maxsize: The configured upper limit of the cache.
- currsize: The current number of stored results.
cache_clear(): Flushes the cache completely, freeing up memory.
```
fibonacci.cache_clear()
```

Best Practices and Limitations

Use Pure Functions Only: The cached function must be deterministic. Given the same inputs, it must always return the same output. Never use lru_cache on functions that depend on database state, network responses, system time, or global variables.
Avoid Side Effects: If a function writes to a file, logs output, or mutates state, those side effects will not execute during a cache hit.
Be Mindful of Memory: Setting maxsize=None on functions with a vast domain of possible inputs can cause uncontrolled memory consumption. Bound the cache with an appropriate maxsize in production environments.