Python defaultdict for Nested Dictionaries

This article explains how Python's collections.defaultdict leverages its factory function to support nested dictionary structures. You will learn the mechanics behind how missing keys trigger recursive or layered factories, how intermediate dictionaries are automatically created (autovivification), and the behavioral nuances between reading and writing nested values.

The Role of the Factory Function

A standard collections.defaultdict takes a callable as its first argument, known as the default_factory. When a requested key does not exist, the dictionary calls its __missing__(key) method under the hood. Instead of raising a KeyError, __missing__ executes default_factory(), assigns the resulting value to the missing key, and returns that value to the caller.

In a flat dictionary, this factory produces simple types like list, set, or int:

from collections import defaultdict

counts = defaultdict(int)
counts["apples"] += 1  # 'apples' is missing -> int() returns 0 -> 0 + 1 = 1

Fixed-Depth Nested Lookups

To handle a predictable two-level dictionary, the factory function can return another defaultdict instance. This is commonly implemented using a lambda:

# A two-level dictionary where the innermost values default to integers
nested = defaultdict(lambda: defaultdict(int))

nested["users"]["alice"] += 5

When Python evaluates nested["users"]["alice"] += 5:

Outer lookup (nested["users"]): The key "users" is missing. The outer defaultdict invokes its default_factory (the lambda), which constructs and returns a new defaultdict(int). This instance is saved under the key "users".
Inner lookup (...["alice"]): Python accesses "alice" on the newly created inner defaultdict(int). Because "alice" is missing, the inner dictionary executes its factory (int()), inserts 0, and adds 5.

Arbitrary-Depth Nesting (Autovivification)

If the depth of the data structure is dynamic or unknown, a self-referential factory function is required. This pattern is often called autovivification:

def tree():
    return defaultdict(tree)

data = tree()
data["country"]["state"]["city"]["population"] = 500000

Here, the function tree acts as its own factory:

Calling tree() produces a defaultdict whose factory is tree.
Evaluating data["country"]["state"]["city"] triggers three successive __missing__ calls.
At each step, a new defaultdict(tree) is instantiated and linked to the parent key.
The final step, ["population"] = 500000, performs a standard key assignment on the innermost dictionary.

The Read Lookup Side Effect

Because defaultdict executes its factory whenever a key is retrieved via the bracket operator ([]), read operations on nonexistent paths will mutate the dictionary by creating intermediate nodes:

data = tree()

# Attempting to inspect a path creates it
_ = data["a"]["b"]["c"]

print(data)
# Output: defaultdict(<function tree at ...>, {'a': defaultdict(<function tree at ...>, {'b': defaultdict(<function tree at ...>, {'c': defaultdict(...)})})})

To inspect values without unintentionally generating nested branches, use the .get() method or standard membership checks (in), which bypass the default_factory entirely.