AFM keeps most performance controls explicit. Start with the default path, measure the workload you care about, then add one control at a time.
General controls
| Control | Useful when | Tradeoff or boundary |
|---|---|---|
--enable-prefix-caching | Agent turns reuse a stable system prompt | Changed token boundaries reduce reuse; the cache stores KV state. |
--kv-bits 4|8 | Long context puts pressure on unified memory | Quantizes KV state; validate quality for the selected model. |
--concurrent N | Multiple sessions share one loaded model | Uses a fair batch scheduler; latency and throughput move differently. |
--gpu-profile | You need request-level device, memory, timing, and bandwidth estimates | Profiling is measurement overhead, not a serving default. |
afm mlx -m <model> \
--enable-prefix-caching \
--concurrent 4 \
--kv-bits 8Speculative decoding
AFM ships two speculative paths, each for one model family. They are not general-purpose speed flags.
| Path | Eligible verifier | Required artifact | Measured result |
|---|---|---|---|
--mtp | Qwen3.6-27B | Checkpoint with mtp.safetensors, such as Youssofal/Qwen3.6-27B-MTPLX-Optimized-Speed | About +52% decode in the repository guide; near-greedy on long output |
--eagle3 <dir> | Dense Gemma4-31B | RedHatAI/gemma-4-31B-it-speculator.eagle3 drafter | About +30% decode; bit-exact greedy output |
# Qwen3.6 with its MTP sidecar
afm mlx -m Youssofal/Qwen3.6-27B-MTPLX-Optimized-Speed --mtp
# Dense Gemma4 with a separate EAGLE3 drafter
afm mlx -m mlx-community/gemma-4-31b-it-4bit \
--eagle3 <eagle3-drafter-directory>Gemma4 MoE 26B-A4B is not supported by the speculative path. A plain Qwen3.6 conversion without the MTP sidecar cannot use --mtp.
Automatic optimizations
- Adaptive long-context SDPA: a build-time MLX patch; the repository reports roughly +10% decode at 16k on the tested Qwen3.6-27B / M4 Pro setup.
- Eager reasoning tag: streaming thinking models emit the opening tag earlier to reduce time to first token.
- Metal prewarm: kernels are prepared at startup to reduce cold first-token cost.
Fixed-schedule DwarfStar
DwarfStar is a separate DeepSeek V4 execution backend, not an MLX optimization toggle. It uses an executor-ready local checkpoint and a fixed Metal schedule; the broad MLX feature set does not carry across the runtime boundary. Read DwarfStar + DeepSeek V4 before converting or serving a checkpoint.
Diagnostics
# Decode throughput, cache activity, tool calls, timing
AFM_DEBUG=1 afm mlx -m <model> --mtp
# EAGLE3 verify/draft phase timings
AFM_DEBUG=1 AFM_EAGLE3_PROFILE=1 afm mlx -m <model> \
--eagle3 <drafter>How to compare runs
- Pin the exact model and weight revision.
- Record temperature, parser, grammar, stream mode, concurrency, and cache state.
- Separate prefill, time to first token, decode throughput, and wall time.
- Validate output before claiming a speedup.
- Do not compare a repaired tool-call run with a native-parser run as if they were identical workloads.