Models + performance
Improves Qwen 3.8 Flash Next, GLM 5.3 Flash, and DeepSeek V4 Flash 0731 with MTP, execution optimizations, sparse verification, and bounded-memory fixes.
Mac only · Apple Silicon · native Swift
Run local AI on your Mac behind an API your tools already understand.
AFM is built exclusively for Apple Silicon Macs. It serves Hugging Face MLX models or Apple's on-device model on macOS, with the protocol details agent clients need: tools, structured output, streaming, cancellation, concurrency, and operational signals.
Mac quick startlocalhost:9999
$ brew install scouzi1966/afm/afm
$ export MACAFM_MLX_MODEL_CACHE=~/.cache/afm
$ afm mlx -m Qwen3-0.6B-4bit -wServing is self-contained in one native executable. Model weights download on first use.
Platform · Mac onlyApple SiliconmacOS 26+
APIOpenAI-compatibleplus local extensions
ServerSwift + Vaporone native executable
InferenceMLX · Apple · DS4choose per workload
What's new
AFM 0.9.19 is now available.
Expanded Qwen, GLM, and DeepSeek support; native checkpoint conversion; OpenAI Responses, Anthropic Messages, and image APIs; stronger cache and runtime reliability; an updated WebUI; and detailed cross-machine qualification evidence.
Open the release archive →Read the qualification report →Native checkpoint conversion
Adds Qwen Next conversion controls, mapped or resident n-gram storage, and safer storage preflight and resume handling.
More API compatibility
Adds OpenAI Responses, Anthropic Messages and token counting, text completions, and image generation and editing for supported backends.
Cache + runtime reliability
Improves prefix-cache validation, batch dispatch, capability reporting, assistant prefill, tool-call collection, rate limiting, and fatal startup errors.
What ships
A server surface, not just a chat demo.
AFM joins model execution, client compatibility, Apple frameworks, and observability in one process.
InferenceMLX language and vision-language modelsApple Foundation ModelsDwarfStar fixed-schedule checkpoints
Agent protocolNative and repaired tool callsStrict JSON with xgrammarStreaming reasoning and tool deltas
OperationsFair concurrent queuesCancellation and request IDsPrometheus and OpenAPI
Apple frameworksVision OCR and document tablesSpeech recognition and synthesisNaturalLanguage embeddings
Choose a runtime
One HTTP contract. Different execution boundaries.
The runtimes are not interchangeable. Pick the one whose constraints match the work.
Compare in the guide →MLX
Open language and vision-language models, full sampling controls, agent tooling, KV controls, speculative decoding, and concurrency.
afm mlx -m <model>Apple Foundation Models
The system model with minimal setup and support for Apple .fmadapter LoRA adapters. The feature set differs from MLX.
afmDwarfStar
A fixed-schedule Metal executor for compatible DeepSeek V4 GGUFs. AFM 0.9.16 resolves and resumes Hugging Face repository downloads directly.
afm mlx -m <owner/repo>Run DwarfStar now →
Gateway
Discover Ollama, LM Studio, Jan, and related local servers and expose their models through one list.
afm --gatewayHTTP surface
Familiar where it should be. Explicit where it differs.
POST
/v1/chat/completionsChat, SSE, tools, JSON schema, reasoning, logprobs
POST
/v1/responsesResponses, previous-response chaining, tools, media
POST
/v1/messagesAnthropic Messages compatibility and streaming
POST
/v1/completionsRaw-prompt text completions on supported runtimes
POST
/v1/images/generationsLocal image generation with supported FLUX models
POST
/v1/images/editsLocal image editing from PNG, JPEG, or WebP input
Operational truths
Important before you benchmark or deploy.
- 01
Set the model cache explicitly. Use
MACAFM_MLX_MODEL_CACHEto avoid the upstream default under~/Documents, iCloud eviction, and TCC prompts. - 02
Grammar enforcement is opt-in. JSON response modes and token-level xgrammar enforcement are different layers; start the server with
--enable-grammar-constraintsfor the latter. - 03
Speculative decode has a narrow fast path. MTP and EAGLE3 require specific checkpoints, greedy text-only requests, and serial execution. Ineligible requests fall back to ordinary decode.
- 04
Parser choice changes tool-call results. Native, repair, and raw modes are deliberate. Record the parser mode when comparing models.
Documentation
Start with the constraint you actually have.
First local endpointInstall, cache, runtime, request →Agent client reliabilityFormats, parser modes, structured output →OCR, speech, embeddingsInputs, response contracts, errors →Throughput and memoryPrefix cache, concurrency, MTP, EAGLE3 →Parameter combinationsRecipes, precedence, incompatibilities →Model compatibilityEvery accepted architecture and alias →Run DwarfStar nowCurrent DeepSeek V4 download and launch command →