AMD has launched the Instinct MI350P PCIe accelerator card, entering existing enterprise servers with an air-cooled, dual-slot PCIe 5.0 form factor, focusing on deploying large model inference without the need for liquid cooling or rebuilding data centers.

The MI350P single card is equipped with 144GB of HBM3E high-bandwidth memory, with a maximum memory bandwidth of 4TB/s, and offers up to 2.3 PFLOPS of dense FP8 matrix computing performance. The 144GB video memory can accommodate FP16 weights for models with approximately 26 billion parameters, or support larger models through quantization, but "single card support for 260 billion parameters" depends on quantization precision, context length, KV Cache, and whether CPU or multi-card offloading is used.

The MI350P is a passive cooling, dual-slot PCIe 5.0 x16 accelerator, positioned by AMD as a product that can be integrated into traditional enterprise servers, avoiding the need for customers to directly enter AI factory-level deployments that rely on liquid cooling, dedicated racks, and high-density networks.

"Token throughput exceeds competitors by four times" is not a conclusion that AMD has publicly disclosed or can independently reproduce for a single card. Liqid claims that its UltraStack 30 system, based on 30 MI350P cards, aims to achieve up to 3.7 times token throughput, 2.1 times tokens per dollar, 1.8 times tokens per watt, and 65% lower deployment costs compared to "traditional multi-server designs"; baseline systems, models, inference engines, batch processing, and measurement methods have not been fully disclosed.

AMD's higher-end route is the Helios rack-level platform: 72 MI455X GPUs can provide up to 2.9 exaFLOPS of FP4 computing power, 31TB of HBM4, 260TB/s scale-up bandwidth, and 43TB/s scale-out bandwidth, aimed at large inference, training, and fine-tuning. The MI350P serves the enterprise local deployment market that cannot or does not wish to enter large-scale liquid-cooled clusters.

In market dynamics, buyers are IT departments in banks, manufacturing, healthcare, government, and large enterprises that wish to run models internally but are constrained by existing data center power, cooling, compliance, and data localization requirements; sellers include AMD, server OEMs, system integrators, inference software, model suppliers, and memory network component manufacturers. The MI350P lowers deployment barriers by being "compatible with existing racks," potentially shifting a portion of enterprise inference budgets from Nvidia's high-density GPU clusters and public cloud APIs to local PCIe servers; however, the maturity of ROCm software, enterprise toolchains, model optimization, channel supply, and real per-token costs will still determine whether it can shake Nvidia's high-end inference market share.

Source: Public information