MTPLX
3x decode TPS increase On Qwen 3.6 27B @ temp 0.6 | Native MTP Speculative Decoding On Apple Silicon With No External Drafter.
What is it?
What it is
MTPLX is a native Mac application and command-line tool designed to run local large language models (LLMs) efficiently on Apple Silicon hardware using multi-token prediction (MTP) techniques.
Why it exists
MTPLX exists to leverage Apple Silicon's hardware capabilities for significantly faster LLM inference compared to traditional methods, enabling real-time, accurate decoding with reduced latency and resource usage.
Who should use it
Apple Silicon Mac users (M1/M2/M3/M5), developers, and AI enthusiasts needing local LLM inference with performance optimization. Ideal for running models like Qwen 3.5/3.6 with multi-token prediction.
Who should avoid it
Users without Apple Silicon hardware, those with <16GB RAM (for larger models), or those avoiding command-line tools. Not suitable for non-technical users requiring minimal setup.
How it works
A quick walkthrough in plain English
How MTPLX works
Step 1 of 3
You interact with it
Open MTPLX, send a request, or connect it to your stack.
Features
Advantages
- Open source (Apache-2.0)
- Active Python ecosystem
- Self-hosted deployment options
Disadvantages
- Requires operational ownership for self-hosted setups
- Community support varies by project maturity
Installation
cloud
Not covered in README
docker
Not covered in README
native
brew install youssofal/mtplx/mtplx tplx start or python3 -m pip install mtplx
compose
Not covered in README
FAQ
What are the system requirements to run MTPLX?
MTPLX requires Apple Silicon (M1 or newer) and macOS 14+. For comfortable operation, 16 GB RAM handles 4B and 9B models, while 27B models need 32 GB or more. The app checks hardware before recommending a model.
How do I install and start MTPLX via the command line?
You can install with Homebrew (`brew install youssofal/mtplx/mtplx`) or pip (`python3 -m pip install mtplx`). Then run `mtplx start` for an interactive setup, or `mtplx serve --port 8000` to launch just the API server.
What does the auto‑tune feature do and how do I run it?
Auto‑tune measures autoregressive decoding versus MTP depths (D1, D2, D3) on your specific Mac, keeping the fastest depth that beats the baseline. Run it with `mtplx tune --model <model-or-path> --retune`; results are saved for future runs.
Can I create my own MTP‑ready models with MTPLX?
Yes. The Forge workflow (`mtplx forge` or the app’s Forge tab) converts a Hugging Face repo to MLX, trains the MTP adapter, verifies speed‑up and exactness on your hardware, and can publish the result back to the Hub.
What API endpoints does the MTPLX server provide and are they compatible with OpenAI/Anthropic clients?
The server (`mtplx start`) serves an OpenAI‑compatible API at `127.0.0.1:8000` (`/v1/chat/completions`, `/v1/completions`, `/v1/models`) and an Anthropic‑compatible `/v1/messages` endpoint, with streaming, tool calls, `/health`, and `/metrics`. Any client that speaks these APIs (OpenAI, anthropic Python packages, Claude Code, Cline, Open WebUI, etc.) works.
How does MTPLX ensure that using MTP heads does not change the model’s output distribution?
MTPLX uses exact rejection sampling based on the Leviathan‑Chen theorem with residual correction. This keeps the output distribution identical to standard decoding for any temperature, top_p, top_k, or penalty settings—only the speed improves.
Featured in Videos
YouTube tutorials and walkthroughs for MTPLX
Alternatives
Similar projects ranked by category, topics, and text overlap.
