Methodology for Benchmarking NAS Throughput

Benchmarking Network Attached Storage (NAS) throughput requires a strict, repeatable methodology to accurately measure how fast data moves between a storage server and client machines. This article outlines the standard process for testing NAS performance, detailing the necessary network environment setup, storage pool configurations, workload types, benchmarking tools, and best practices for eliminating caching anomalies to yield reliable hardware evaluations.

1. Standardizing the Test Environment

To isolate the NAS as the sole performance variable, the client computer and network infrastructure must exceed the maximum theoretical performance of the NAS.

  • Client Hardware: Use a high-end client PC equipped with a fast PCIe 4.0 or 5.0 NVMe SSD. This ensures the client's local storage does not bottleneck data transfers during real-world copy tests.
  • Network Infrastructure: Connect the NAS to the client using enterprise-grade switches and appropriate cabling (Cat6a, Cat7, or SFP+ fiber) capable of handling 2.5GbE, 10GbE, 25GbE, or 100GbE connections. To eliminate switch overhead entirely during baseline testing, use a direct attached connection between the client and the NAS.
  • Network Settings: Standardize the Maximum Transmission Unit (MTU) settings. Test with standard frames (1500 MTU) first, and if testing 10GbE or faster, conduct a secondary test pass using Jumbo Frames (9000 MTU) if all hardware supports it.
  • OS Optimization: Disable antivirus software, firewalls, and background indexing on the client machine to prevent artificial network latency.

2. Configuring the NAS and Storage Pool

Throughput heavily depends on how the storage drives and operating system are configured. Every variable must be documented and kept consistent across comparative tests.

  • Drive Selection: Use identical drives (matching brand, model, capacity, and firmware) for the storage pool.
  • RAID Configuration: Benchmark using standard, real-world RAID levels. For 4-bay to 8-bay systems, RAID 5 and RAID 6 are the industry standards for testing parity calculation overhead. Use RAID 10 for high-performance comparisons. Wait for the initial RAID synchronization and parity scrubbing to completely finish before running any tests.
  • File System: Document the file system used (e.g., EXT4, Btrfs, or ZFS), as features like inline data compression and checksums impact throughput.
  • SSD Caching: If the NAS supports NVMe SSD caching, conduct two completely separate benchmark runs: one with the mechanical hard drives alone to find the baseline performance, and another with the read/write cache enabled.
  • Protocols: Test over the specific network protocols relevant to the target audience. SMB (Server Message Block) is standard for Windows and macOS environments, NFS is used for Linux, and iSCSI is used for block-level virtualization workloads.

3. Overcoming the RAM Cache Trap

Modern NAS operating systems utilize unused system RAM to cache incoming and outgoing data. If a benchmark test file fits entirely within the NAS's RAM, the benchmark will measure the speed of the system memory and network interface rather than the actual storage drives.

To measure true storage throughput, the test file size must be significantly larger than the total installed RAM of the NAS. The standard methodology requires a test data set that is at least two to four times the size of the NAS memory. For example, if a NAS has 16GB of RAM, the benchmark test file size must be at least 32GB to 64GB.

4. Defining Test Workloads

A comprehensive throughput methodology evaluates how the NAS handles different types of data movement.

  • Sequential Read and Write: This measures the speed of transferring large, contiguous files (such as 4K video files or system backups). It is tested using large block sizes (typically 1MB) and is measured in Megabytes per second (MB/s) or Gigabytes per second (GB/s).
  • Random Read and Write: This measures performance with small, non-contiguous files (such as databases or operating system files) using 4K block sizes. While typically measured in IOPS (Input/Output Operations Per Second), recording the resulting MB/s throughput shows how severely small files degrade transfer speeds.
  • Mixed Workloads: Real-world server environments rarely do 100% reads or 100% writes. Configure a test with a 70% read / 30% write split to simulate active office or virtualization environments.
  • Queue Depths and Threads: Test with a single thread and a Queue Depth of 1 (QD1T1) to simulate a single user transferring a file. Follow this with a high-concurrency test (such as Queue Depth 8 with 8 Threads) to simulate multiple users accessing the NAS simultaneously.

5. Selecting Benchmarking Tools

Use a combination of synthetic benchmarking software and real-world file transfers to gather a complete performance profile.

  • FIO (Flexible I/O Tester): The gold standard for storage benchmarking. It is scriptable, cross-platform, and allows exact control over block sizes, queue depths, test duration, and caching behavior.
  • IOmeter: A robust alternative to FIO for Windows environments that allows deep customization of synthetic network workloads.
  • CrystalDiskMark: Useful for quick, standardized sequential and random throughput checks over a mapped SMB network drive. Ensure the test size is set to the maximum (64GiB) to bypass RAM caching on higher-end NAS units.
  • Real-World File Copy (Robocopy): Synthetic tests represent best-case scenarios. Validate them by using a command-line tool like Windows Robocopy to transfer two real data sets: a single 100GB video file (sequential test) and a 50GB folder containing 50,000 small documents and photos (random metadata test).

6. Test Execution and Data Analysis

How the tests are run is just as important as the tools used.

  • Sustained Testing: Run each synthetic workload for a minimum of 3 to 5 minutes. This ensures that any drive-level caches (like SLC caching on SSDs) are exhausted and you are measuring sustained throughput.
  • Cool-Down Periods: Allow the NAS to sit idle for 5 to 10 minutes between tests. This lets the operating system flush its buffers, complete background garbage collection, and prevents thermal throttling on NVMe drives.
  • Multiple Passes: Run every test at least three times. Discard obvious anomalies caused by sudden network hiccups and average the remaining results.
  • Hardware Monitoring: Actively monitor the NAS's internal resource monitor during testing. Record the CPU and RAM utilization alongside the MB/s throughput scores. If throughput plateaus while the NAS CPU hits 100% utilization, you have identified a processor bottleneck (common with heavy encryption or RAID 6 parity calculations) rather than a drive or network limit.