Wafer, an AI inference cloud provider, published a detailed blog post today on how they served GLM-5.2 on AMD’s MI355X (Instinct MI350 series) — hitting 2,626 tok/s/node aggregated throughput at saturation, with a p50 TTFT of 0.81s. At the same workload, that’s about 80% of a B200’s performance, at over 2x lower cost per GPU. The single-stream number is equally respectable: 213 tok/s on a 10k-in/1.5k-out workload, following Artificial Analysis standards.
The engineering behind it is a lesson in AMD’s current position in AI inference: the silicon is competitive, but the software stack still requires work. Wafer quantized GLM-5.2 from bf16 to MXFP4 using AMD’s Quark tool — and the results were essentially lossless against the FP8 baseline across GSM8K, GPQA-Diamond, and tau2. They chose sglang over vLLM (no working MXFP4 + MoE path) and ATOM (output degraded at long context). Then came the interesting part: speculative decode on the ROCm build didn’t work out of the box. Two fixes — a module prefix mismatch in the MTP head’s shared expert, and an unguarded #include <cuda_runtime.h> in a fused kernel — and spec dec unlocked close to 3x single-stream throughput.
🎩 Cask’s Take
The numbers are impressive, but the story here is really about the gap closing. For months, the narrative around AMD in AI has been “great silicon, terrible software” — and while the software gap is real, each of these engineering reports shrinks it by a measurable amount. Two patches. That’s all it took to unlock speculative decode on a flagship Chinese model running on AMD hardware. The fact that Wafer can serve GLM-5.2 at 2,626 tok/s for roughly half the Blackwell cost per token is exactly the kind of data point that makes cloud architects rethink their GPU procurement strategy. The token supply shortage isn’t going away, and AMD is the only credible alternative at scale. Every one of these blog posts moves the needle.