{"id":136248,"date":"2026-07-14T15:00:20","date_gmt":"2026-07-14T15:00:20","guid":{"rendered":"https:\/\/blogs.nvidia.com\/?p=96103"},"modified":"2026-07-14T15:00:20","modified_gmt":"2026-07-14T15:00:20","slug":"why-performance-per-watt-is-the-ultimate-metric-for-ai-infrastructure-efficiency-2","status":"publish","type":"post","link":"http:\/\/webdomino.net\/index.php\/nvidia\/why-performance-per-watt-is-the-ultimate-metric-for-ai-infrastructure-efficiency-2\/","title":{"rendered":"Why Performance per Watt Is the Ultimate Metric for AI Infrastructure Efficiency"},"content":{"rendered":"
<\/div>

Power is AI infrastructure\u2019s inescapable constraint. How many <\/span>tokens<\/span><\/a> an AI factory can generate within a fixed power budget determines its revenue and profitability. Because of this, performance per watt \u2014 a metric that can\u2019t be gamed, only earned through real-world results \u2014 is the foundation for AI factories.\u00a0<\/span><\/p>\n

As agentic AI drives token demand higher, the infrastructure decisions organizations make today will determine who scales and who doesn\u2019t in a power-constrained world.<\/span><\/p>\n

Virtually every frontier AI model today runs on a <\/span>mixture-of-experts<\/span><\/a> (MoE) architecture. Serving MoE at rack scale demands codesign across every layer of the system and software stack, plus the operational depth earned from running these models under real production load. With the <\/span>NVIDIA Blackwell NVL72 platform<\/span><\/a>, that <\/span>rack-scale <\/span>foundation is already built and proven, delivering the highest performance per watt to maximize revenues and the lowest token cost to maximize profit margins. It\u2019s this foundation that the <\/span>NVIDIA Vera Rubin<\/span><\/a> platform builds upon next to further elevate rack-scale energy efficiency.<\/span><\/p>\n

Maximizing Performance per Watt for Frontier AI\u00a0<\/b><\/h2>\n

Each new generation of frontier models brings architectural changes that unlock greater intelligence while demanding new optimizations to run efficiently at scale.\u00a0<\/span><\/p>\n

Across the newest generation of leading open models, NVIDIA GB300 NVL72 delivers up to 25x performance per watt compared with the NVIDIA Hopper generation. These numbers reflect where Blackwell stands today, a starting point that continues to improve.\u00a0<\/span><\/p>\n

Any single number only tells part of the story. Different workloads demand different operating points: some optimize for latency, others for throughput and cost \u2014 and most need to move between the two.\u00a0<\/span><\/p>\n

To best represent these operating points, NVIDIA showcases Pareto curves for each model rather than a single point and provides tools such as <\/span>DynoSim<\/span><\/a> to help teams find their optimal point on the Pareto frontier before spending a single GPU-hour on validation.<\/span><\/p>\n

\"\"
NVIDIA GB300 NVL72 systems deliver up to 25x performance per watt over NVIDIA Hopper on DeepSeek V4 Pro. Source: SemiAnalysis InferenceX<\/figcaption><\/figure>\n
\"\"
On GLM5.1 NVIDIA GB300 NVL72 systems deliver up to 20x performance per watt over NVIDIA Hopper. Source: SemiAnalysis InferenceX<\/figcaption><\/figure>\n
\"\"
NVIDIA GB300 NVL72 systems deliver up to 10x performance per watt over NVIDIA Hopper for Kimi K2.6, a model purpose-built for long-horizon agentic tasks. Source: SemiAnalysis InferenceX<\/figcaption><\/figure>\n

The performance per watt NVIDIA Blackwell delivers is a result of extreme codesign: every component of the rack-scale system, from silicon to <\/span>software<\/span><\/a>, designed together to maximize token throughput for AI inference workloads. That codesign touches every layer of the stack.\u00a0\u00a0<\/span><\/p>\n

For example, <\/span>NVIDIA NVLink Switch<\/span><\/a>, critical for rack-scale performance, is purpose-built for scale-up GPU domains, not adapted from general-purpose networking. Now in its sixth generation with the Vera Rubin platform, its capabilities are designed specifically for AI workloads such as SHARP, which performs in-network computing directly in the switch, offloading work from the GPUs themselves.<\/span><\/p>\n

NVIDIA\u2019s <\/span>inference software stack<\/span><\/a>, including NVIDIA Dynamo and TensorRT LLM, as well as SGLang and vLLM, is built to run the full range of optimizations: NVFP4 quantization, disaggregated serving, large-scale expert parallelism, KV-aware routing, KV cache offloading and more. These stack together to multiply the performance each GPU delivers. Moreover, software keeps improving performance over time: On DeepSeek V4, performance per watt improved by up to 5x in a single month.<\/span><\/p>\n

In AI factories, power lost to cooling and rack-level inefficiencies can mean only about 60% of the electricity pulled from the grid turns into useful AI work. NVIDIA DSX MaxLPS, the power-and-efficiency software in the <\/span>NVIDIA DSX<\/span><\/a> platform, closes that gap by shifting power between GPUs and racks in real time, supporting warm-water liquid cooling and using techniques like power steering to wring more performance. This enables operators to run up to 40% more GPUs within the same power budget.<\/span><\/p>\n

Production Is Where It Counts<\/b><\/h2>\n

Rack-scale reliability at AI factory scale is hard-won. Rack-scale systems introduce failure modes that single-node deployments never encounter, and handling them requires engineering rigor and time in production.<\/span><\/p>\n

NVIDIA Blackwell NVL72 systems continues to set the standard across a diverse range of models and production use cases delivering sustained performance, rack-level reliability and economics that hold under real traffic day after day.\u00a0<\/span><\/p>\n

That\u2019s why leading AI labs such as <\/span>Anthropic and OpenAI <\/span>use NVIDIA Blackwell NVL72 systems to run inference.<\/span><\/p>\n

In addition, a variety of inference service providers and AI natives use the Blackwell platform to deploy open models in production.<\/span><\/p>\n

CoreWeave <\/span>has deployed Kimi K2.6<\/span><\/a> on NVIDIA GB300 NVL72, combining NVFP4 quantization and EAGLE3 speculative decoding to maximize inference performance.\u00a0<\/span><\/p>\n

Perplexity<\/span> runs <\/span>Qwen3 235B<\/span> <\/a>and post-trained Qwen3.5-397B-A17B<\/span> on NVIDIA GB200 NVL72 for its AI agent platform, serving millions of queries daily with the latency and reliability that consumers need.<\/span><\/p>\n

Fireworks AI<\/span> deploys GLM 5.2 on the NVIDIA Blackwell platform, enabling production deployments for customers including Cursor and Factory AI.<\/span><\/p>\n

This accumulated production experience, built across generations of frontier models and real-world deployments, is what gives NVIDIA Vera Rubin its head start.<\/span><\/p>\n

Learn more about the NVIDIA Vera Rubin platform in this <\/span><\/i>technical blog<\/span><\/i><\/a> and find details on the <\/span><\/i>NVIDIA DSX AI factory-scale platform and DSX MaxLPS<\/span><\/i><\/a>.<\/span><\/i><\/p>\n","protected":false},"excerpt":{"rendered":"

Power is AI infrastructure\u2019s inescapable constraint. How many tokens an AI factory can generate within a fixed power budget determines its revenue and profitability. Because of this, performance per watt \u2014 a metric that can\u2019t be gamed, only earned through real-world results \u2014 is the foundation for AI factories.\u00a0 As agentic AI drives token demand [\u2026]<\/p>\n","protected":false},"author":593,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_mi_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[343],"tags":[13704,1360,12142,244,13696,13710,52,13692],"yoast_head":"\nWhy Performance per Watt Is the Ultimate Metric for AI Infrastructure Efficiency - WebDomino.NET<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"http:\/\/webdomino.net\/index.php\/nvidia\/why-performance-per-watt-is-the-ultimate-metric-for-ai-infrastructure-efficiency-2\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Why Performance per Watt Is the Ultimate Metric for AI Infrastructure Efficiency - WebDomino.NET\" \/>\n<meta property=\"og:description\" content=\"Power is AI infrastructure\u2019s inescapable constraint. How many tokens an AI factory can generate within a fixed power budget determines its revenue and profitability. Because of this, performance per watt \u2014 a metric that can\u2019t be gamed, only earned through real-world results \u2014 is the foundation for AI factories.\u00a0 As agentic AI drives token demand [\u2026]\" \/>\n<meta property=\"og:url\" content=\"http:\/\/webdomino.net\/index.php\/nvidia\/why-performance-per-watt-is-the-ultimate-metric-for-ai-infrastructure-efficiency-2\/\" \/>\n<meta property=\"og:site_name\" content=\"WebDomino.NET\" \/>\n<meta property=\"article:published_time\" content=\"2026-07-14T15:00:20+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/blogs.nvidia.com\/wp-content\/uploads\/2026\/07\/nvidia-blackwell-delivers-25x-throughput-per-watt.png\" \/>\n<meta name=\"author\" content=\"Shruti Koparkar\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Shruti Koparkar\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"http:\/\/webdomino.net\/index.php\/nvidia\/why-performance-per-watt-is-the-ultimate-metric-for-ai-infrastructure-efficiency-2\/\",\"url\":\"http:\/\/webdomino.net\/index.php\/nvidia\/why-performance-per-watt-is-the-ultimate-metric-for-ai-infrastructure-efficiency-2\/\",\"name\":\"Why Performance per Watt Is the Ultimate Metric for AI Infrastructure Efficiency - WebDomino.NET\",\"isPartOf\":{\"@id\":\"https:\/\/webdomino.net\/#website\"},\"datePublished\":\"2026-07-14T15:00:20+00:00\",\"dateModified\":\"2026-07-14T15:00:20+00:00\",\"author\":{\"@id\":\"https:\/\/webdomino.net\/#\/schema\/person\/06026ff9c63144b03cbc604543fada5c\"},\"breadcrumb\":{\"@id\":\"http:\/\/webdomino.net\/index.php\/nvidia\/why-performance-per-watt-is-the-ultimate-metric-for-ai-infrastructure-efficiency-2\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"http:\/\/webdomino.net\/index.php\/nvidia\/why-performance-per-watt-is-the-ultimate-metric-for-ai-infrastructure-efficiency-2\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"http:\/\/webdomino.net\/index.php\/nvidia\/why-performance-per-watt-is-the-ultimate-metric-for-ai-infrastructure-efficiency-2\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/webdomino.net\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Why Performance per Watt Is the Ultimate Metric for AI Infrastructure Efficiency\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/webdomino.net\/#website\",\"url\":\"https:\/\/webdomino.net\/\",\"name\":\"WebDomino.NET\",\"description\":\"Global NEWS WordPress site\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/webdomino.net\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/webdomino.net\/#\/schema\/person\/06026ff9c63144b03cbc604543fada5c\",\"name\":\"Shruti Koparkar\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/webdomino.net\/#\/schema\/person\/image\/\",\"url\":\"http:\/\/1.gravatar.com\/avatar\/a9aa30766444dc889f6deae4df3cf4eb?s=96&d=mm&r=g\",\"contentUrl\":\"http:\/\/1.gravatar.com\/avatar\/a9aa30766444dc889f6deae4df3cf4eb?s=96&d=mm&r=g\",\"caption\":\"Shruti Koparkar\"},\"sameAs\":[\"https:\/\/nvidianews.nvidia.com\"],\"url\":\"http:\/\/webdomino.net\/index.php\/author\/shruti-koparkar\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Why Performance per Watt Is the Ultimate Metric for AI Infrastructure Efficiency - WebDomino.NET","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"http:\/\/webdomino.net\/index.php\/nvidia\/why-performance-per-watt-is-the-ultimate-metric-for-ai-infrastructure-efficiency-2\/","og_locale":"en_US","og_type":"article","og_title":"Why Performance per Watt Is the Ultimate Metric for AI Infrastructure Efficiency - WebDomino.NET","og_description":"Power is AI infrastructure\u2019s inescapable constraint. How many tokens an AI factory can generate within a fixed power budget determines its revenue and profitability. Because of this, performance per watt \u2014 a metric that can\u2019t be gamed, only earned through real-world results \u2014 is the foundation for AI factories.\u00a0 As agentic AI drives token demand [\u2026]","og_url":"http:\/\/webdomino.net\/index.php\/nvidia\/why-performance-per-watt-is-the-ultimate-metric-for-ai-infrastructure-efficiency-2\/","og_site_name":"WebDomino.NET","article_published_time":"2026-07-14T15:00:20+00:00","og_image":[{"url":"https:\/\/blogs.nvidia.com\/wp-content\/uploads\/2026\/07\/nvidia-blackwell-delivers-25x-throughput-per-watt.png"}],"author":"Shruti Koparkar","twitter_misc":{"Written by":"Shruti Koparkar","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"http:\/\/webdomino.net\/index.php\/nvidia\/why-performance-per-watt-is-the-ultimate-metric-for-ai-infrastructure-efficiency-2\/","url":"http:\/\/webdomino.net\/index.php\/nvidia\/why-performance-per-watt-is-the-ultimate-metric-for-ai-infrastructure-efficiency-2\/","name":"Why Performance per Watt Is the Ultimate Metric for AI Infrastructure Efficiency - WebDomino.NET","isPartOf":{"@id":"https:\/\/webdomino.net\/#website"},"datePublished":"2026-07-14T15:00:20+00:00","dateModified":"2026-07-14T15:00:20+00:00","author":{"@id":"https:\/\/webdomino.net\/#\/schema\/person\/06026ff9c63144b03cbc604543fada5c"},"breadcrumb":{"@id":"http:\/\/webdomino.net\/index.php\/nvidia\/why-performance-per-watt-is-the-ultimate-metric-for-ai-infrastructure-efficiency-2\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["http:\/\/webdomino.net\/index.php\/nvidia\/why-performance-per-watt-is-the-ultimate-metric-for-ai-infrastructure-efficiency-2\/"]}]},{"@type":"BreadcrumbList","@id":"http:\/\/webdomino.net\/index.php\/nvidia\/why-performance-per-watt-is-the-ultimate-metric-for-ai-infrastructure-efficiency-2\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/webdomino.net\/"},{"@type":"ListItem","position":2,"name":"Why Performance per Watt Is the Ultimate Metric for AI Infrastructure Efficiency"}]},{"@type":"WebSite","@id":"https:\/\/webdomino.net\/#website","url":"https:\/\/webdomino.net\/","name":"WebDomino.NET","description":"Global NEWS WordPress site","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/webdomino.net\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/webdomino.net\/#\/schema\/person\/06026ff9c63144b03cbc604543fada5c","name":"Shruti Koparkar","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/webdomino.net\/#\/schema\/person\/image\/","url":"http:\/\/1.gravatar.com\/avatar\/a9aa30766444dc889f6deae4df3cf4eb?s=96&d=mm&r=g","contentUrl":"http:\/\/1.gravatar.com\/avatar\/a9aa30766444dc889f6deae4df3cf4eb?s=96&d=mm&r=g","caption":"Shruti Koparkar"},"sameAs":["https:\/\/nvidianews.nvidia.com"],"url":"http:\/\/webdomino.net\/index.php\/author\/shruti-koparkar\/"}]}},"_links":{"self":[{"href":"http:\/\/webdomino.net\/index.php\/wp-json\/wp\/v2\/posts\/136248"}],"collection":[{"href":"http:\/\/webdomino.net\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/webdomino.net\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/webdomino.net\/index.php\/wp-json\/wp\/v2\/users\/593"}],"replies":[{"embeddable":true,"href":"http:\/\/webdomino.net\/index.php\/wp-json\/wp\/v2\/comments?post=136248"}],"version-history":[{"count":6,"href":"http:\/\/webdomino.net\/index.php\/wp-json\/wp\/v2\/posts\/136248\/revisions"}],"predecessor-version":[{"id":136309,"href":"http:\/\/webdomino.net\/index.php\/wp-json\/wp\/v2\/posts\/136248\/revisions\/136309"}],"wp:attachment":[{"href":"http:\/\/webdomino.net\/index.php\/wp-json\/wp\/v2\/media?parent=136248"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/webdomino.net\/index.php\/wp-json\/wp\/v2\/categories?post=136248"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/webdomino.net\/index.php\/wp-json\/wp\/v2\/tags?post=136248"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}