NVMe vs SAS vs SATA Storage for AI Servers — Protocol, Performance, and Cost

Enterprise servers connect storage using one of three protocols: NVMe (Non-Volatile Memory Express), SAS (Serial Attached SCSI), or SATA (Serial ATA). These protocols determine maximum throughput, latency, queue depth, and ultimately whether storage keeps pace with GPU compute or becomes the bottleneck. For AI workloads — training dataset loading, checkpoint writing, model weight serving — the choice between NVMe and SAS has a direct impact on GPU utilisation.

The Three Protocols Explained

NVMe is a storage access protocol designed specifically for solid-state NAND flash. It connects directly to the server's PCIe bus, bypassing the legacy HBA (Host Bus Adapter) stack that SAS requires. NVMe U.2 drives (2.5" enterprise form factor) and NVMe E1.S drives (data centre optimised, denser) connect via PCIe Gen4 or Gen5 lanes directly to the CPU or a PCIe switch. The key architectural advantage: NVMe supports up to 65,535 I/O queues with 65,535 commands each (vs SAS's single 254-command queue), which eliminates queueing latency under concurrent I/O from multiple GPU processes.

SAS is a mature enterprise storage protocol running over a point-to-point serial connection from server to drive, via an SAS HBA or expander. SAS supports both HDDs and SSDs. SAS SSDs are enterprise-grade with strong endurance ratings and have been the default enterprise flash choice since the mid-2010s. SAS 24Gbps (the current generation) provides 24 Gbps per lane; most SAS SSDs achieve 2–4 GB/s sequential read throughput. SAS backplanes support hot-plug, dual-path access (two SAS controllers for redundancy), and disk expanders for high-density JBOD shelves.

SATA is the lowest-bandwidth storage protocol: 6 Gbps per lane (SATA III), theoretical maximum ~600 MB/s per drive. SATA is found on HDDs and low-cost consumer/SMB SSDs. Enterprise server storage rarely uses SATA SSD — the performance ceiling is too low for most workloads beyond cold archive. SATA HDDs are used for bulk, low-access-frequency object storage and backup tiers.

Performance Comparison

• NVMe U.2 PCIe Gen4 (e.g., Samsung PM9A3, Kioxia CD8): Sequential read 6–7 GB/s, sequential write 3–4 GB/s, random read 4KIOPS 1–1.5M IOPS, latency 70–120 μs
• NVMe U.2 PCIe Gen5 (e.g., Samsung PM1743, Micron 9550): Sequential read 14 GB/s, sequential write 7 GB/s, random read 4K IOPS 2.5–3M IOPS, latency 50–80 μs
• SAS SSD 24G (e.g., Seagate Nytro 3350, Samsung PM1643a): Sequential read 2–4 GB/s, sequential write 1–2 GB/s, random read 4K IOPS 250–500K IOPS, latency 100–200 μs
• SAS HDD 12G (e.g., Seagate Exos X20, Western Digital Ultrastar): Sequential read 250–285 MB/s, random read 4K IOPS 200–300 IOPS, latency 3–5 ms
• SATA SSD (e.g., Samsung 870 EVO, Micron 5400): Sequential read ~550 MB/s, latency 80–150 μs; interface-limited

NVMe PCIe Gen5 delivers 35–50× more sequential bandwidth than a SAS HDD and 3–7× more than a SAS SSD. The IOPS gap is larger still: NVMe Gen5 delivers 10× more random 4K IOPS than a SAS SSD due to queue depth and protocol efficiency.

Which Protocol for AI Training Storage

AI training datasets — tokenised text, image datasets, video frames — are read sequentially in large chunks during DataLoader prefetching. The GPU compute rate determines the minimum required storage bandwidth: if 8× H100 GPUs can process a training batch in 200 ms, the DataLoader must deliver the next batch within that 200 ms window. For a 4 TB ImageNet-scale dataset with 256 KB average file size, this requires sustained 8–15 GB/s read bandwidth to avoid GPU starvation.

NVMe is the correct protocol for AI training storage. A single 1U server with 8× NVMe U.2 PCIe Gen4 drives provides up to 48 GB/s aggregate sequential read bandwidth — more than enough for any single GPU server's training data loading requirement. SAS SSDs, at 2–4 GB/s per drive, would require 12–24 drives to achieve the same bandwidth and would still be limited by the SAS HBA's aggregate bandwidth ceiling (typically 24–48 GB/s for dual-port HBA configurations, but with higher CPU overhead).

Which Protocol for AI Checkpoint Storage

Training checkpoints — saved model weights written to disk periodically during training — require high sequential write bandwidth. A LLaMA-70B checkpoint is approximately 140 GB at BF16. With 8× H100 GPUs, the training framework wants to save a checkpoint every 1,000 steps, which at 30 steps/second is every 33 seconds. Completing a 140 GB write in 33 seconds requires 4.2 GB/s sustained write bandwidth.

NVMe handles this easily; SAS SSD does too (at the cost of more drives); SAS HDD cannot sustain 4 GB/s write without a large RAID stripe. Most AI cluster deployments use a local NVMe RAID-0 array for checkpoint writes, then replicate to shared object storage (MinIO, Ceph, or cloud) asynchronously for durability.

SAS HDD Use Cases in AI Infrastructure

Despite lower performance, SAS HDDs remain relevant in AI infrastructure for:

• Cold dataset storage: Raw training data archives (petabyte-scale text, image, video corpora) where access frequency is low (load once for preprocessing, then cache preprocessed tokens on NVMe). Cost per TB for enterprise SAS HDD (~$20–30/TB) vs NVMe ($100–200/TB) is 5–8× lower.
• Long-term checkpoint archive: Training checkpoints older than 30 days, retained for reproducibility but rarely accessed, appropriate for SAS HDD JBOD shelves.
• Bulk object storage (Ceph OSD): Ceph object storage clusters typically use SAS HDD for capacity tiers and NVMe SSD for journal/WAL metadata drives. A 60-bay 4U JBOD with 20 TB SAS HDDs holds 1.2 PB raw per shelf.

NVMe-oF: Extending NVMe Across the Network

NVMe-oF (NVMe over Fabrics) extends NVMe protocol semantics across a network — either InfiniBand (using RDMA) or 100G/400G Ethernet (using RoCEv2 or TCP). This allows a cluster of GPU servers to share a centralised NVMe storage array with latency close to local NVMe (200–500 μs vs 70–120 μs local). NetApp AFF A-Series, Pure Storage FlashArray//C, and VAST Data Universal Storage are examples of shared NVMe-oF all-flash arrays used in AI cluster deployments.

Haink has supplied NVMe-oF storage configurations for sovereign cloud and research institution deployments in the Gulf and Hong Kong regions, typically pairing 400G Ethernet leaf-spine with VAST Data or NetApp all-flash arrays as shared training storage.

Decision Summary

• AI training data loading (active dataset): NVMe PCIe Gen4/Gen5 — mandatory for GPU cluster throughput
• AI checkpoint write: NVMe (local) → async replicate to shared storage
• Shared cluster training storage: NVMe-oF (VAST, NetApp AFF, Pure Storage) over 100G/400G Ethernet
• Cold dataset archive (>30 days): SAS HDD JBOD — cost-optimised at $20–30/TB
• Enterprise transactional (databases, ERP): SAS SSD 24G — mature, redundant, dual-path, predictable
• Edge inference server (low power, compact): NVMe M.2 PCIe Gen4 — adequate bandwidth, small form factor

Related Resources

• Storage Systems — Haink Supply
• NetApp vs Pure Storage — Which for AI?
• Storage Supplier
• NVLink vs PCIe GPU Interconnect
• Case Study: Sovereign Cloud Storage Architecture
• Case Study: Research Institution Storage Network

Frequently Asked Questions

Is NVMe better than SAS for databases?

For databases with high random IOPS requirements (OLTP, high-frequency trading, real-time analytics), NVMe PCIe Gen4/Gen5 outperforms SAS SSD by 3–10× in IOPS and delivers lower latency. However, enterprise SAS SSDs offer advantages in mission-critical deployments: mature dual-path redundancy (two independent SAS controllers), universal compatibility with existing SAS backplanes, and proven 5-year endurance ratings. For new AI-first deployments, NVMe is the correct default; for extending an existing SAS infrastructure, high-endurance SAS SSD remains viable.

Can I mix NVMe and SAS in the same server?

Yes. Most enterprise server platforms (HPE ProLiant DL380 Gen11, Dell PowerEdge R760, Supermicro SYS-221H) support mixed NVMe and SAS drive bays using a tri-mode storage controller (e.g., Broadcom 9560-16i or HPE MR216i-p) that handles NVMe, SAS, and SATA simultaneously. A typical AI server configuration uses NVMe U.2 drives for the active training data cache and SAS HDDs (or a SAS JBOD shelf) for cold archive. The OS boot drive is typically an M.2 NVMe module separate from the data bays.

What storage does NVIDIA DGX H100 use?

The NVIDIA DGX H100 includes 8× 3.84 TB NVMe U.2 drives (PCIe Gen4, ~30 TB raw) in a direct-attached RAID-0 configuration providing approximately 25 GB/s aggregate read bandwidth — sized to sustain data loading for 8× H100 GPUs in parallel. For larger training datasets exceeding the DGX local storage capacity, DGX systems connect to shared NVMe-oF storage (VAST, NetApp, NVIDIA Base Command cluster storage) over 400G InfiniBand or Ethernet.