Why Lodash sortedUniqBy Is Faster Than uniqBy

In the Lodash JavaScript library, _.sortedUniqBy delivers significantly higher performance than standard uniqueness utilities like _.uniqBy by taking advantage of pre-sorted data. While general uniqueness functions must allocate extra memory and track every previously seen item, _.sortedUniqBy only compares contiguous elements in a single linear pass. This architectural difference eliminates the need for hash lookups, reduces memory allocation, and minimizes garbage collection overhead, making it drastically faster when operating on sorted arrays.

Algorithmic Differences and Time Complexity

Standard uniqueness functions like _.uniq and _.uniqBy assume the input array is unsorted. To identify duplicates, these functions must maintain a lookup table—typically a JavaScript Set or Lodash's internal SetCache structure—to record every unique value encountered. As the algorithm iterates through the collection, it executes the iteratee function and checks whether the resulting computed key already exists in the set. While Set.prototype.has provides average \(O(1)\) lookup time, hashing values and maintaining dynamic hash structures introduce significant CPU constant factors.

In contrast, _.sortedUniqBy operates under the strict assumption that the incoming array is already ordered according to the criteria defined by the iteratee. Because identical values are guaranteed to be adjacent to one another, the algorithm does not need to maintain a global history of seen values. It only needs to compare the transformed value of the current item with the transformed value of the most recently retained item. Both approaches operate in \(O(n)\) linear time, but _.sortedUniqBy executes far fewer CPU instructions per element.

Minimal Memory Allocation

The primary bottleneck in _.uniqBy during large-scale operations is memory allocation. Creating and resizing a Set or hash map to store up to \(n\) elements places substantial pressure on the JavaScript engine's heap memory. This leads to frequent garbage collection cycles, which pause execution and degrade overall throughput.

_.sortedUniqBy requires zero auxiliary storage for tracking seen items. It utilizes a single variable to retain the last computed value. By eliminating hash-map allocations, _.sortedUniqBy operates with \(O(1)\) auxiliary space complexity (excluding the output array). This lightweight footprint avoids memory bloat and keeps the JavaScript runtime focused solely on iteration and comparison.

CPU Cache Locality

Because _.sortedUniqBy processes the array sequentially without jumping into external hash-table memory locations, it benefits heavily from CPU cache locality. Modern processors excel at reading contiguous blocks of memory. A simple comparison between adjacent values keeps the relevant data inside the CPU's L1/L2 cache lines, drastically lowering latency compared to the scattered memory reads inherent in hash-table lookups.

Practical Application

The performance gains of _.sortedUniqBy only apply if the dataset is already sorted. If an unsorted array must first be ordered using an \(O(n \log n)\) sorting algorithm simply to use _.sortedUniqBy, the combined operation will often be slower than running _.uniqBy directly. However, when working with data retrieved in sorted order—such as rows returned from a database query with an ORDER BY clause or ordered time-series events—_.sortedUniqBy is the optimal choice for removing duplicates with maximum speed and minimal memory consumption.