AI Engine on Small Unified Memory brings a 125B model to PS5 Linux
A new streaming inference engine runs the 125B Qwen3.8-Flash-Next model on PS5 Linux using Vulkan compute and M.2 SSD streaming.
A developer has released AI Engine on Small Unified Memory (LaRazaDelSol/AIEngineOnSmallUniMem), a streaming inference engine written in C++20 and GLSL to run the Qwen3.8-Flash-Next 125B mixture-of-experts model on a PlayStation 5 running Linux. The project targets the console’s 16 GB of unified memory and Vulkan compute on GFX1013, streaming 24,576 experts directly from an M.2 NVMe SSD.
Tested on a PS5 Phat on firmware 13.00 via ps5-linux, the engine achieves 49 to 57 tokens per second during decode using cache-aware routing, and up to 340 tokens per second during prefill. It manages a 262,144-token context window and restores a 120,000-token conversation in about one second.
Running it requires an unupdated console with an existing Linux installation, a Gen4 NVMe SSD with around 400 GB free, and specific console packages like shaderc and vulkan-headers. Pinning memory past the pool reserve can hang the console.