AMD Launches Air-Cooled MI350P to Capture Enterprise Inference
AMD has launched the Instinct MI350P PCIe accelerator card, entering existing enterprise servers with an air-cooled, dual-slot PCIe 5.0 form factor, focusing on deploying large model inference without the need for liquid cooling or rebuilding data centers.
The MI350P single card is equipped with 144GB of HBM3E high-bandwidth memory, with a maximum memory bandwidth of 4TB/s, and offers up to 2.3 PFLOPS of dense FP8 matrix computing performance. The 144GB video memory can accommodate FP16 weights for models with approximately 26 billion parameters, or support larger models through quantization, but "single card support for 260 billion parameters" depends on quantization precision, context length, KV Cache, and whether CPU or multi-card offloading is used.
The MI350P is a passive cooling, dual-slot PCIe 5.0 x16 accelerator, positioned by AMD as a product that can be integrated into traditional enterprise servers, avoiding the need for customers to directly enter AI factory-level deployments that rely on liquid cooling, dedicated racks, and high-density networks.
"Token throughput exceeds competitors by four times" is not a conclusion that AMD has publicly disclosed or can independently reproduce for a single card. Liqid claims that its UltraStack 30 system, based on 30 MI350P cards, aims to achieve up to 3.7 times token throughput, 2.1 times tokens per dollar, 1.8 times tokens per watt, and 65% lower deployment costs compared to "traditional multi-server designs"; baseline systems, models, inference engines, batch processing, and measurement methods have not been fully disclosed.
AMD's higher-end route is the Helios rack-level platform: 72 MI455X GPUs can provide up to 2.9 exaFLOPS of FP4 computing power, 31TB of HBM4, 260TB/s scale-up bandwidth, and 43TB/s scale-out bandwidth, aimed at large inference, training, and fine-tuning. The MI350P serves the enterprise local deployment market that cannot or does not wish to enter large-scale liquid-cooled clusters.
In market dynamics, buyers are IT departments in banks, manufacturing, healthcare, government, and large enterprises that wish to run models internally but are constrained by existing data center power, cooling, compliance, and data localization requirements; sellers include AMD, server OEMs, system integrators, inference software, model suppliers, and memory network component manufacturers. The MI350P lowers deployment barriers by being "compatible with existing racks," potentially shifting a portion of enterprise inference budgets from Nvidia's high-density GPU clusters and public cloud APIs to local PCIe servers; however, the maturity of ROCm software, enterprise toolchains, model optimization, channel supply, and real per-token costs will still determine whether it can shake Nvidia's high-end inference market share.
Source: Public information
ABAB AI Insight
The competition between AMD and Nvidia has long been not just a battle of single-card peak performance, but a contest of software ecosystems, supply chains, and data center forms. Nvidia has upgraded GPUs from server components to a complete AI factory standard relying on CUDA, DGX, HGX, NVLink, and networking products; while AMD's early Instinct products, despite having high bandwidth memory and computing capabilities, have faced long-term issues with ROCm compatibility, developer tools, and cluster integration. The MI350P's choice of PCIe and air cooling avoids directly replicating Nvidia's path in ultra-large liquid-cooled clusters, instead targeting the upgrade needs of existing enterprise servers.
From a capital perspective, enterprise AI deployment typically requires a choice between "buying GPUs and building in-house" and "calling cloud model APIs." High-end liquid-cooled clusters require purchasing racks, networking, power, cooling, GPUs, storage, and maintenance services, leading to significant capital expenditures; the MI350P attempts to narrow the procurement focus to accelerator cards and supporting software that can be inserted into existing servers. If customers can run medium-scale models locally at lower transformation costs, their spending will shift from ongoing cloud token fees to one-time hardware depreciation, software support, and internal maintenance, benefiting server OEMs, enterprise integrators, and edge AI software vendors.
A historical analogy is the diffusion path of x86 servers defeating dedicated minicomputers. Enterprises do not always purchase the most powerful systems, but prioritize systems that can be absorbed by existing data centers, power supplies, maintenance personnel, procurement processes, and application stacks. The MI350P's strategy is to downgrade AI computing power from "new AI factory" to "regular server upgrade component"; this is closer to the popularization of enterprise GPUs rather than challenging the highest performance records of training clusters. Its challenge is that large model inference is limited not only by memory but also by model quantization, KV Cache, interconnect, concurrent requests, and software kernel efficiency.
Essentially, this represents a restructuring of the industry chain. AI infrastructure has previously concentrated among cloud vendors with liquid-cooled data centers, power access, and large-scale GPU procurement capabilities; if air-cooled PCIe cards can provide sufficient inference cost-effectiveness, they will redistribute some computing power back to enterprise data centers, industry clouds, and local system integrators. The technical threshold shifts from "can the largest model be trained" to "can useful models be reliably, compliantly, and cost-effectively run on existing assets." However, as long as the software stack and developer tools remain dominated by Nvidia, hardware compatibility alone is insufficient to achieve real market share migration.
ABAB News · Cognitive Laws
Enterprises are not buying GPUs, but capabilities that do not require reconstruction.
Memory determines whether models can run, software determines whether they can make money.
The endpoint of AI diffusion is not the data center but the existing data center.