How Python Implements Arbitrary Keyword Arguments

This article explores the internal mechanisms Python uses to handle arbitrary keyword arguments using the **kwargs syntax. It covers how the CPython compiler identifies variable keywords, sets internal code flags, processes arguments during execution, and constructs the resulting dictionary dynamically at runtime.

When you define a function containing a **kwargs parameter, Python's compiler recognizes the double asterisk as a directive to accept an arbitrary number of keyword arguments. The identifier itself does not have to be named kwargs, though community convention adheres strictly to it; the syntax is defined solely by the ** prefix.

During the compilation of the function into bytecode, the compiler inspects the parameter list. When it encounters the ** token, it sets a specific bitwise flag on the function's underlying code object (code.co_flags). This flag is CO_VARKEYWORDS (represented numerically as 0x08). The presence of CO_VARKEYWORDS signals to the Python Virtual Machine (PVM) that this function can accept keyword arguments that are not explicitly defined in the function signature.

At runtime, when the function is invoked, argument parsing and evaluation occur via CPython's internal function-calling routines (such as _PyFunction_Vectorcall or _PyEval_Vector). CPython processes the arguments in a defined sequence:

  1. Positional Arguments: Explicit positional arguments are bound to their corresponding local parameter slots.
  2. Explicit Keyword Arguments: Passed keyword arguments that match explicitly declared positional-or-keyword or keyword-only parameters are assigned to their respective local variable slots.
  3. Keyword Collection: Any remaining keyword arguments that do not match explicitly named parameters are directed to the variable keyword collector.

If the CO_VARKEYWORDS flag is set and excess keyword arguments were passed, CPython allocates a new standard Python dictionary (PyDictObject). It populates this dictionary with the leftover keyword names as string keys and their corresponding argument values. If no excess keyword arguments were provided in the function call, CPython allocates an empty dictionary.

This newly created dictionary is then assigned to the local variable slot reserved for kwargs. Within the function's execution frame, kwargs behaves strictly as a regular Python dictionary, offering standard methods like .keys(), .values(), and .items().

On the caller side, using ** (such as func(**data)) triggers dictionary unpacking. The bytecode compiler emits instructions (traditionally BUILD_MAP_UNPACK_WITH_CALL or handled directly in modern versions via CALL_FUNCTION_EX) that evaluate the mapping object, verify it is a valid mapping, and unpack its key-value pairs into the argument evaluation stack before passing them to the callee's parameter resolution logic.