N3komata Brings SSD-Backed MoE Inference to the AMD BC-250
N3komata is a new standalone release for running large Mixture-of-Experts models on the AMD BC-250 board using Vulkan and memory mapping.
Developer nyacom released N3komata, an SSD-backed inference engine tailored for the AMD BC-250 board, a Linux gaming PC built with a cut-down PlayStation 5 APU and unified memory. The project targets Mixture-of-Experts (MoE) models that exceed the system’s 16 GiB memory limit by keeping sharded GGUF files on storage and streaming active weights through Vulkan and memory mapping.
Running models larger than available RAM requires balancing storage reads and cache capacity. N3komata builds on llama.cpp and introduces an expert cache, dynamic K routing, and batched uploads to handle cache misses. When the router selects an expert missing from the Vulkan cache, the engine loads the weight range from memory-mapped files.
The standalone release includes prefetch options and a release profile utilizing router-mass selection to manage the hardware constraints of the BC-250 platform, which features a narrow PCIe 2.0 x2 SSD link. Cold misses still wait on storage speed. Warm misses can be served from the OS page cache, which competes with Vulkan allocations for the same memory. The project calls itself experimental and does not make a 70 GiB model resident in 16 GiB.