Combining AI Checkpoints via Training-Free Model Merging

This article explores how model merging techniques combine multiple fine-tuned artificial intelligence checkpoints into a single, capable model without requiring additional training. By manipulating the mathematical weights of neural networks directly, methods like Spherical Linear Interpolation (SLERP), TIES-Merging, and DARE bypass the need for expensive GPU clusters and large datasets. Below is a straightforward breakdown of the mechanics behind weight-space manipulation and how these specific algorithms resolve conflicts between different AI models to merge their distinct capabilities.

The Foundation: Weight Space and Task Vectors

At its core, a neural network checkpoint is a massive collection of numbers, known as weights or parameters, organized into matrices. When two models are fine-tuned from the same base model, their weights remain aligned in the same mathematical space.

Instead of retraining a model to learn multiple skills, model merging operates on "task vectors." A task vector is the mathematical difference between a fine-tuned model's weights and the original base model's weights. This vector represents the exact adjustments the model made to learn a specific skill, such as coding or creative writing. Merging techniques manipulate and combine these task vectors, applying the final result back to the base model.

Spherical Linear Interpolation (SLERP)

Basic model merging uses linear averaging, which simply adds the weights of two models together and divides by two. However, in the high-dimensional space of neural networks, a straight line between two points often passes through areas of lower magnitude, which can degrade the model's performance.

Spherical Linear Interpolation (SLERP) solves this by treating the model weights as points on a hypersphere. Instead of taking a straight path through the interior of the sphere, SLERP calculates an arc along the surface.

  • Preserves Magnitude: By following the curve of the hypersphere, SLERP maintains the geometric properties and the overall magnitude of the weights.
  • Two-Model Limit: SLERP is mathematically designed to interpolate between two checkpoints at a time, using a blend factor (e.g., 70% Model A and 30% Model B). To merge more than two models, SLERP must be applied sequentially in a tree-like structure.

TIES-Merging: Trim, Elect, and Merge

When merging multiple task vectors simultaneously, models often interfere with one another. One model might increase a specific weight to improve math skills, while another decreases that same weight to improve language translation. TIES-Merging (Trim, Elect Sign, and Merge) resolves this parameter interference through a three-step process:

  1. Trim: During fine-tuning, many weight changes are tiny and represent random noise rather than useful learning. TIES resets these small changes back to zero, keeping only the top percentage (usually 20%) of the largest, most impactful weight adjustments in each task vector.
  2. Elect Sign: For every specific weight across all the models being merged, TIES checks whether the adjustments are positive or negative. It sums the magnitudes of the positive changes and compares them to the negative changes. The direction with the highest total magnitude wins, and becomes the "elected sign" for that weight.
  3. Disjoint Merge: Finally, the algorithm averages the values for each weight, but it only includes the models whose weight adjustments match the elected sign. Models that tried to push the weight in the opposite direction are ignored for that specific parameter.

DARE: Drop And REscale

DARE (Drop And REscale) is a preprocessing technique designed to drastically reduce redundancy in task vectors before merging them. Research shows that fine-tuned models are heavily over-parameterized; most weight changes are not strictly necessary for the model to retain its new capabilities.

DARE operates in two simple steps:

  1. Drop: The algorithm randomly selects a massive percentage of the task vector changes—often between 90% and 99%—and resets them to zero. This reverts those specific weights back to their original base model values.
  2. Rescale: Simply dropping 90% of the changes would weaken the model's newly learned skill. To compensate, DARE multiplies the remaining, non-zero weights by a scaling factor (Calculated as \(1 / (1 - p)\), where \(p\) is the drop rate). If 90% of weights are dropped, the remaining 10% are multiplied by 10.

This rescaling preserves the mathematical expectation of the layer outputs. Because DARE leaves the task vectors mostly empty (sparse), multiple DARE-processed models can be combined using simple addition or TIES-Merging with minimal risk of the models overwriting each other's weights.

Summary of Differences

While all three methods combine checkpoints without gradient descent or training data, they serve different roles in the merging pipeline:

  • SLERP is best for smoothly blending the foundational characteristics of two distinct models while preserving weight geometry.
  • DARE acts as a filter, stripping away redundant weight changes so multiple models can coexist in the same parameter space.
  • TIES-Merging acts as a conflict resolution system, actively sorting out directional disagreements when combining many specialized task vectors at once.