How to Choose NVIDIA GPUs for AI — a Practical Buying Guide

Written and maintained by Haink's AI infrastructure team · Updated July 2026 · every GPU order export-screened

Haink supplies NVIDIA AI and data center hardware to enterprises, research institutions, cloud providers, and AI teams in Hong Kong, Dubai, and Mainland China. The NVIDIA portfolio available through Haink includes data center GPU platforms (H100, H200, B200, B300), NVIDIA L40S inference GPUs, NVIDIA RTX Ada Generation professional GPUs for workstations, NVIDIA DGX personal and enterprise AI systems, and NVIDIA InfiniBand and Ethernet networking for high-performance AI clusters.

NVIDIA is the dominant supplier of AI training and inference compute globally, with H100 and H200 installed in the majority of the world's largest AI data centers and H100 SXM5 being the most widely deployed GPU for large language model training. NVIDIA's CUDA ecosystem — the software platform that runs PyTorch, TensorFlow, and all major AI frameworks — has a 10+ year head start over competing GPU platforms, making NVIDIA GPUs the default choice for AI infrastructure in most enterprise and research deployments.

Choosing an NVIDIA GPU by Workload

Start from the workload, not the model number. This picks the right class of GPU; the catalog below has the exact platforms.

What ships vs what waits (as of July 2026): Hopper (H100/H200), L40S and DGX Spark are generally available from stock; Blackwell (B200/B300, GB200/GB300 NVL72) is allocation-constrained — order early. The next architecture, Vera Rubin (VR200 NVL72), reaches volume availability in H2 2026. Prior generations (A100, V100) are legacy — and A100-class GPUs are export-restricted to some destinations.

How to Decide — a 5-Step Framework

1. Model size & precision — estimate the memory footprint (parameters × bytes/param, plus KV cache). A 70B model at FP8 needs ~70 GB for weights alone; long-context serving adds tens of GB of KV cache. This sets the minimum GPU memory and how many GPUs per replica.
2. Training or inference — training and fine-tuning want SXM (NVLink, HBM bandwidth); inference is usually cheaper on fewer PCIe GPUs (L40S, RTX PRO 6000 Blackwell) unless the model is memory-bound, where H200/B200 earn their price.
3. Scale — one node (≤8 GPUs) or multi-node? Multi-node changes the build: you now need an InfiniBand or Spectrum-X fabric matched to the GPU generation, and the network — not the GPU — often becomes the bottleneck.
4. Facility — can the data center actually power and cool it? Blackwell density usually forces direct-liquid cooling (see below). This constrains the GPU choice more often than budget does.
5. Own vs rent — high, sustained utilization favours owning; short or spiky demand favours renting. We model the break-even against your real workload rather than a list price.

How Much GPU Do You Actually Need? The Memory Math

Most oversizing comes from guessing. The dominant cost is GPU memory, and it is largely predictable. Rule of thumb: weight memory (GB) ≈ parameters (billions) × bytes per parameter — 1 byte at FP8/INT8, 0.5 at FP4/INT4, 2 at FP16. Then add the KV cache for inference: roughly 30–60% more at long context and high concurrency.

Worked example — serving Llama-3-70B at FP8, 8K context, ~50 concurrent sessions: weights ≈ 70 GB; KV cache ≈ 40–55 GB → ~110–125 GB total. That is tight on a single H200 (141 GB), comfortable on one B200 (192 GB), or about 4× L40S (48 GB) with 4-bit weights. The same model for a handful of users fits on a single H200 or two L40S.

Figures are planning rules of thumb, not guarantees — real footprint depends on context length, batch size, quantization and framework. We size it precisely for your model and target throughput.

Facility Reality: Power, Cooling & Fabric

The GPUs are the easy part; the building is where projects stall. Rough numbers to plan around (as of July 2026):

• Power density — an 8× H100 SXM node draws roughly 10 kW; an 8× B200 node ~14+ kW; a GB200 / GB300 NVL72 rack is ~120+ kW. Many enterprise data halls are provisioned for 8–15 kW per rack, so a single Blackwell rack can exceed a whole row's power budget.
• Cooling — air cooling is practical to roughly 30–40 kW per rack; B200/B300 at density and every NVL72 rack require direct-liquid cooling (DLC). No DLC on site narrows your GPU options before price does.
• Fabric — match the network to the GPU generation: NDR 400G for H100/H200, 800G XDR / Spectrum-X for B200/B300, Quantum-X800 with ConnectX-9 for GB300 and Rubin. An undersized fabric silently caps training throughput.

Haink sizes power, cooling and fabric with the GPUs, not after — see our AI infrastructure cost guide and reference architectures.

Own vs Rent — and When NVIDIA Isn't the Answer

We stay neutral on the verdict, because getting it wrong is expensive. Honestly:

• Rent (cloud GPU) if demand is short, spiky or exploratory — a POC, a one-off fine-tune, or utilization you can't hold above roughly half. Idle owned H100s are the most common way to overspend.
• Own if utilization is high and sustained (typically 12+ months of steady work) or your data can't leave the network. At real utilization the break-even against cloud usually lands inside a year — we'll model it for your workload.
• Consider AMD Instinct (MI325X / MI355X) for memory-bound inference on a ROCm-friendly stack: 256–288 GB HBM per GPU can beat NVIDIA on cost-per-GB.
• Wait for the next generation if your timeline allows and you don't need capacity now — Vera Rubin reaches volume availability in H2 2026.

See cloud vs private AI for the full own-vs-rent break-even logic.

Data Center GPUs — Hopper Generation

NVIDIA H100

• NVIDIA H100 SXM5 80GB — the most widely deployed AI training GPU; 3,958 TFLOPS FP8, 3.35 TB/s memory bandwidth, NVLink 4.0 (900 GB/s bidirectional), 700W TDP; installed in DGX H100, Supermicro SYS-821GE, Dell XE9680, HPE Cray XD670 8-GPU SXM server platforms
• NVIDIA H100 PCIe 80GB — 3,958 TFLOPS FP8, 2 TB/s memory bandwidth, PCIe Gen5, 350W TDP; installed in standard PCIe server platforms (Supermicro SYS-420GP, SYS-220HE, SYS-111E) for inference clusters and training workloads where NVLink bandwidth is not the bottleneck
• NVIDIA H100 NVL — dual H100 on a single PCIe card with NVLink bridge providing 188 GB combined VRAM for single-card inference of very large models

NVIDIA H200

• NVIDIA H200 SXM5 141GB — same GH100 Hopper die as H100 with 141 GB HBM3e (vs H100's 80 GB HBM3); 4.8 TB/s memory bandwidth; identical FP8 compute to H100; primary use case is inference of 70B+ models that require more than 80 GB GPU memory, and memory-bandwidth-bound training workloads
• NVIDIA H200 PCIe NVL 141GB — PCIe form factor H200 for inference servers; enables single-GPU serving of 70B models that previously required two H100 PCIe cards

Data Center GPUs — Blackwell Generation

NVIDIA B200

• NVIDIA B200 SXM 192GB — 9,000 TFLOPS FP8 / 18,000 TFLOPS FP4, 8 TB/s HBM3e memory bandwidth, NVLink 5.0 (1,800 GB/s bidirectional), ~1,000W TDP; Blackwell architecture delivers 2.3× more FP8 compute than H100 SXM5; requires direct liquid cooling at full utilization; the current-generation primary GPU for new large-scale AI training cluster deployments
• New in Blackwell: FP4 precision (first in NVIDIA data center GPU line), NVLink 5.0 with 2× H100 bandwidth, RAS Engine for proactive error detection, Confidential Computing hardware memory encryption

NVIDIA B300 (Blackwell Ultra)

• NVIDIA B300 SXM — 288 GB HBM3e (50% more than B200), higher FP8 throughput than B200, NVLink 5.0; DLC mandatory; designed for frontier model pre-training at 1T+ parameter scale and highest-concurrency inference of large deployed models
• Available in the same 8-GPU SXM server form factor as B200 (Supermicro ARS-821GL-NHR and equivalent platforms)

NVIDIA GB200 NVL72

• NVIDIA GB200 NVL72 — rack-scale architecture combining 36 Grace CPU modules and 72 B200 GPU dies in a single NVLink 5.0 domain; 130 TB/s total NVLink fabric bandwidth; the entire rack operates as a single unified compute domain for model parallelism; requires full rack direct liquid cooling; designed for training frontier models and serving the largest deployed models at hyperscale; available through NVIDIA DGX GB200 NVL72 rack systems
• NVIDIA GB300 NVL72 — Blackwell Ultra rack-scale system: 72 B300 GPUs (288 GB HBM3e each, 8 TB/s) and 36 Grace CPUs in one NVLink domain, ~50% more FP4 compute than GB200 NVL72; fully liquid-cooled, shipping from early 2026

NVIDIA L40S — Inference and Visualization

• NVIDIA L40S 48GB — Ada Lovelace architecture, 48 GB GDDR6 ECC, 362 TFLOPS FP8, PCIe Gen4 passive cooling (server form factor); designed for AI inference, video transcoding, and professional visualization in rackmount servers; the bridge between professional workstation GPUs (RTX Ada) and data center training GPUs (H100)
• L40S does not require liquid cooling, fits in standard PCIe server slots, and provides substantially more inference throughput than RTX 6000 Ada at lower cost than H100; optimal for AI inference servers serving 20–200 concurrent users on 7B–34B models
• Deployed in Supermicro SYS-111E (2× L40S 1U), SYS-221GE (4× L40S 2U), and standard PCIe servers from Dell and HPE

NVIDIA RTX PRO Blackwell — Inference and Professional GPUs

The RTX PRO Blackwell generation is the current professional line. The RTX PRO 6000 Blackwell Server Edition (96 GB GDDR7, passively cooled for servers) is the new standard for high-density inference nodes, while the RTX PRO 6000 / 5000 / 4500 / 4000 Blackwell Workstation Edition cards cover on-desk AI development and visualization. The prior RTX Ada generation (below) remains available as a value / from-stock option.

• NVIDIA RTX PRO 6000 Blackwell Server Edition — 96 GB GDDR7, PCIe Gen5, MIG support; the current inference-node GPU (more memory than L40S for new builds)
• NVIDIA RTX PRO 6000 / 5000 / 4500 / 4000 Blackwell Workstation Edition — professional workstation GPUs for AI development, 3D and CAD

NVIDIA RTX Ada Generation — Professional Workstation GPUs (Value / From Stock)

• NVIDIA RTX 4000 Ada Generation — 20 GB GDDR6 ECC, entry professional GPU for AI workstations, 3D design, and CAD workflows; supports local inference of 7B–13B models
• NVIDIA RTX 4500 Ada Generation — 24 GB GDDR6 ECC, mid-range professional GPU; improved performance over RTX 4000 Ada for AI workstation and design workloads
• NVIDIA RTX 5000 Ada Generation — 32 GB GDDR6 ECC, PCIe 4.0 x16; runs 34B models at quantization for small team AI inference workstations
• NVIDIA RTX 6000 Ada Generation — 48 GB GDDR6 ECC, the highest-VRAM RTX Ada workstation GPU; runs 70B models at 4-bit quantization on a single card; two RTX 6000 Ada in NVLink provides 96 GB for higher-quality 70B inference; Haink's primary recommendation for AI workstation deployments requiring maximum local LLM capability
• NVIDIA RTX 5000 Ada and RTX 6000 Ada both support NVLink bridging for dual-GPU configurations in tower workstations

NVIDIA DGX Systems

NVIDIA DGX Spark

• NVIDIA DGX Spark — personal AI supercomputer powered by NVIDIA GB10 Grace Blackwell Superchip; 128 GB unified LPDDR5X memory shared between Grace ARM CPU and Blackwell GPU; 1 PFLOPS FP8 AI compute; compact desktop form factor; standard 100–240V power; ships with NVIDIA AI Enterprise stack (NIM, CUDA, TensorRT-LLM) pre-installed; runs 70B models at full FP16 precision; recommended for individuals and small teams running local LLMs, RAG, and AI development without data center infrastructure

NVIDIA DGX H100

• NVIDIA DGX H100 — 8× NVIDIA H100 SXM5 80 GB GPUs, dual Intel Xeon Platinum 8480C CPUs, 2 TB DDR5 system RAM, 8× ConnectX-7 400G InfiniBand, 30 TB NVMe storage; factory-integrated NVIDIA-validated AI training appliance; the reference platform for H100-based AI training infrastructure

NVIDIA DGX H200

• NVIDIA DGX H200 — 8× NVIDIA H200 SXM5 141 GB GPUs in the same DGX chassis as DGX H100; drop-in upgrade providing 76% more GPU memory for inference of larger models and memory-bandwidth-bound training workloads

NVIDIA DGX B200

• NVIDIA DGX B200 — 8× NVIDIA B200 SXM 192 GB GPUs, Grace CPU modules, NVLink 5.0 interconnect, ConnectX-8 InfiniBand networking; next-generation DGX platform delivering 2.3× more FP8 compute than DGX H100; requires direct liquid cooling infrastructure

NVIDIA DGX GB200 NVL72

• NVIDIA DGX GB200 NVL72 — complete liquid-cooled rack system containing 36 GB200 Superchip modules (72 B200 GPUs + 36 Grace CPUs); 130 TB/s NVLink 5.0 fabric; factory-integrated, pre-cabled, and pre-configured by NVIDIA; the flagship product for AI training cluster deployments at hyperscale
• NVIDIA DGX B300 — 8× Blackwell Ultra (B300) GPUs, 288 GB HBM3e each; the current DGX training and inference appliance, factory-integrated (typical lead time 8–12 weeks)
• NVIDIA DGX Station (GB300) — desktop Grace-Blackwell Ultra workstation with a single GB300 and large coherent memory, for on-desk model development
• Coming next — NVIDIA Vera Rubin (VR200 NVL72): the architecture after Blackwell, with 288 GB HBM4 per GPU, 88-core Vera CPUs, ConnectX-9 and NVLink 6; in production with volume availability in H2 2026 (roadmap as of July 2026)

NVIDIA InfiniBand Networking

NVIDIA InfiniBand is the dominant interconnect for AI training clusters, providing GPU-to-GPU communication bandwidth for distributed training across nodes. InfiniBand's RDMA (Remote Direct Memory Access) capability allows GPUs in different servers to communicate directly without CPU involvement, reducing communication overhead during all-reduce operations in distributed LLM training.

• NVIDIA ConnectX-7 400G InfiniBand — 400 Gbps HDR200/NDR200 InfiniBand HCA; installed in DGX H100 and DGX H200 systems; the standard InfiniBand NIC for H100/H200 cluster nodes
• NVIDIA ConnectX-8 800G InfiniBand — 800 Gbps NDR InfiniBand HCA; installed in DGX B200 and next-generation GPU clusters
• NVIDIA QM9700 / QM9790 InfiniBand switches — 64-port NDR 400G InfiniBand switches for building fat-tree or dragonfly InfiniBand fabrics for AI training clusters
• NVIDIA QM8790 HDR 200G InfiniBand switches — for H100 cluster InfiniBand fabrics
• NVIDIA Quantum-X800 — next-generation XDR InfiniBand (144 ports of 800 Gb/s) with ConnectX-8 / ConnectX-9 SuperNICs and SHARP v4 in-network computing, for GB300 NVL72 and Rubin-class trillion-parameter clusters
• NVIDIA Spectrum-X800 — 800G Ethernet AI fabric (Spectrum SN5600 switch + BlueField-3 SuperNIC) for multi-tenant AI clouds that prefer Ethernet to InfiniBand
• Fabric choice follows GPU generation: NDR (QM9700, ConnectX-7) for H100/H200; ConnectX-8 800G for B200/B300; Quantum-X800 XDR / Spectrum-X800 with ConnectX-9 for GB300 and Rubin
• NVIDIA Spectrum-X — Ethernet-based alternative to InfiniBand for AI training clusters using RoCEv2 (RDMA over Converged Ethernet); provides InfiniBand-class performance over standard 400G and 800G Ethernet infrastructure

NVIDIA Software Stack

NVIDIA hardware value is inseparable from the CUDA software ecosystem — the primary reason NVIDIA maintains its AI infrastructure dominance:

• CUDA — parallel computing platform and API running all major AI frameworks (PyTorch, TensorFlow, JAX); 10+ years of optimization and third-party library development
• cuDNN — deep neural network library providing GPU-accelerated primitives for convolution, normalization, and activation operations
• TensorRT — inference optimization and deployment SDK; converts trained models to optimized engines for production inference
• TensorRT-LLM — inference optimization library specifically for large language models; enables multi-GPU tensor parallelism and FP8/FP4 quantized inference for LLMs
• NVIDIA NIM (NVIDIA Inference Microservices) — pre-packaged, optimized inference containers for major LLMs (Llama, Mistral, Nemotron) ready for production deployment
• NCCL (NVIDIA Collective Communications Library) — GPU-to-GPU communication library for distributed training all-reduce operations over InfiniBand and Ethernet

Where Haink Supplies NVIDIA Hardware

• Hong Kong — NVIDIA GPU servers, DGX systems, L40S, RTX Ada workstation GPUs, and InfiniBand networking delivered duty-free through Hong Kong's free port. NVIDIA GPU supplier Hong Kong →
• Dubai — NVIDIA AI infrastructure delivered through Dubai free trade zone logistics for UAE enterprise and data center deployments, with onward distribution to MENA. NVIDIA GPU supplier Dubai →
• Mainland China — NVIDIA GPU hardware availability for Mainland China is subject to current US export control regulations; certain high-performance GPUs (H100, H200, B200) are restricted for export to China. Haink advises on compliant GPU server configurations available for Mainland China delivery. GPU server supplier Mainland China →

Related Resources

• Supermicro — Primary NVIDIA GPU Server Platform
• Dell — PowerEdge XE9680 AI Server
• HPE — Cray XD670 AI Server
• H100 vs H200 vs B200 vs B300 Comparison
• AI Server Supplier
• GPU Infrastructure
• AI Workstation for Small Teams
• Enterprise AI Infrastructure
• All IT Hardware Brands

Five Expensive Mistakes We See in GPU Procurement

1. Buying SXM for inference. Eight-GPU SXM training servers are wasted on serving workloads — L40S or RTX PRO 6000 Blackwell deliver better cost-per-token and do not need liquid cooling.
2. Undersizing the fabric. A cluster is only as fast as its slowest link; NDR-class networking on a Blackwell build throttles the GPUs you paid for. Match the fabric to the generation.
3. No power or cooling plan. Ordering B200/GB200 into an air-cooled hall with 10 kW racks stalls the project on day one. Confirm DLC and power budget before you order, not after.
4. Buying at peak allocation. Committing to the newest GPU at the tightest allocation means paying the most for the longest wait. Often the prior generation ships from stock this month at a better price.
5. Chasing gray-market “bargains.” Re-marked or non-warranted boards are the most expensive way to save. Verify serials and channel before payment (next).

Authorized Channel & Verifying Your Hardware Before Payment

GPUs are a prime target for gray-market and counterfeit supply. Haink sources NVIDIA hardware through authorized distribution only, and every unit is serial-verifiable before you pay. On request we provide the GPU and system serial numbers so you can confirm — before payment — that the hardware is genuine, factory-new (or explicitly identified as refurbished), carries valid manufacturer warranty for your region, and matches the ordered SKU. Combined with per-order export screening (end-user, end-use and destination), this protects you from counterfeit boards, re-marked GPUs, and non-warrantable gray-market stock — the most common risks in fast-moving GPU procurement.

Frequently Asked Questions

Which NVIDIA GPU should I choose — training or inference?

For large-scale training and fine-tuning, use SXM systems: H100/H200 SXM5 (Hopper) or B200/B300 SXM (Blackwell) in 8-GPU servers such as the Supermicro SYS-821GE-TNHR (8× H100/H200) or ARS-821GL-NHR (B200). For inference and model serving, the L40S (48 GB Ada) or the newer RTX PRO 6000 Blackwell Server Edition (96 GB) in standard PCIe servers are more cost-effective and don't need liquid cooling. H200's 141 GB is the memory-bound sweet spot for serving 70B+ models. Not sure? Describe the model and target throughput and we'll size it.

What are realistic lead times, and how does GPU allocation work?

In-stock items (H200 NVL cards, L40S, DGX Spark) ship within days from Hong Kong or Dubai stock. SXM systems are typically 4–10 weeks for H100, 6–12 for H200, 12–20 for B200 and 14–24 for B300 — Blackwell is allocation-constrained, so order early to lock a slot. We confirm current allocation and a delivered lead time for your exact configuration, shipping via Hong Kong (free port, no duty) or the Dubai free trade zone, and every GPU order is export-screened.

What is the difference between NVIDIA H100 and B200?

H100 (Hopper) delivers 3,958 TFLOPS FP8 with 80 GB HBM3 and NVLink 4.0. B200 (Blackwell) delivers 9,000 TFLOPS FP8 / 18,000 TFLOPS FP4 with 192 GB HBM3e and NVLink 5.0 — 2.3× more FP8 compute, 2.4× more memory, 2× more NVLink bandwidth. B200 also introduces FP4 precision and requires direct liquid cooling at full utilization. See the full H100 vs H200 vs B200 comparison.

What is NVIDIA L40S and when should I use it instead of H100?

NVIDIA L40S is a 48 GB GDDR6 Ada Lovelace GPU for AI inference and professional visualization in standard PCIe rackmount servers. L40S does not require liquid cooling and costs substantially less than H100 per GPU. It is the right choice for AI inference serving (deploying trained models to users) rather than AI training. A 1U server with two L40S GPUs (96 GB total VRAM) can serve 70B models at 4-bit quantization to small-to-medium teams at lower infrastructure cost than H100 SXM servers. For large-scale training, H100 or B200 SXM is required.

What is NVIDIA DGX Spark and who is it for?

NVIDIA DGX Spark is a personal AI supercomputer powered by the GB10 Grace Blackwell Superchip with 128 GB unified memory, capable of running 70B parameter models at full FP16 precision in a compact desktop form factor. It is designed for AI researchers, developers, and small teams who need serious local AI compute without data center infrastructure. DGX Spark plugs into a standard power outlet and ships with the complete NVIDIA AI software stack pre-installed.

Can NVIDIA GPUs be exported to Mainland China?

US export control regulations restrict export of certain high-performance NVIDIA GPUs to Mainland China, including H100, H200, A100, and similar data center GPUs above specific performance thresholds. NVIDIA has developed China-specific variants (H20, L20, L2) with reduced performance to comply with export regulations. Haink advises on currently compliant GPU server configurations available for Mainland China delivery on a per-inquiry basis, as regulations and available configurations change.

What InfiniBand switches does Haink supply for AI clusters?

Haink supplies NVIDIA QM9700 and QM9790 NDR 400G InfiniBand switches for H100 and B200 AI training cluster fabrics, and NVIDIA QM8790 HDR 200G switches for existing H100 HDR cluster deployments. InfiniBand switch procurement is coordinated alongside GPU server platform procurement for complete AI training cluster builds.