QuarkStar q36 Local Inference Engine Adds MTP Support for Distributed MoE Models
QuarkStar q36 adds MTP support for distributed inference and MoE models, targeting AMD BC-250 and Apple Silicon.
The native local inference engine QuarkStar (q36) for Qwen3.6-35B-A3B and Qwen3.8-27B has received an update adding MTP support for distributed inference alongside distributed execution for Mixture of Experts models.
Designed for memory-constrained setups, the tool runs on Vulkan-capable Linux hardware, with optimization tuned for the AMD BC-250 board, and on Apple Silicon via Metal. It has a self-contained graph runner, HTTP server, and coding agent capabilities without relying on general-purpose GGUF runners. The Vulkan fast path was tuned on the BC-250 with RADV, and the generic backend is untested on other GPUs.
How to try it
- Clone the repository from GitHub.
- For Linux with Vulkan, make sure the GPU supports Vulkan 1.1 or newer with shader float64 and int64, or follow the hardware-specific setup for the BC-250.
- For Apple Silicon (M1 or newer, macOS 11 or later), run
make metaland execute./q36 --metalto launch the native Metal graph runtime.