AFM — local AI infrastructure for Apple Silicon

POST /v1/chat/completions is the primary interoperability surface. The JSON shape follows OpenAI conventions, while AFM exposes extra local controls and runtime measurements where they are useful.

A complete structured request

curl http://127.0.0.1:9999/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -H 'X-Request-ID: agent-plan-42' \
  -d '{
    "model": "mlx-community/Qwen3.5-4B-4bit",
    "messages": [
      {"role": "system", "content": "Answer as a precise local assistant."},
      {"role": "user", "content": "Return two deployment risks as JSON."}
    ],
    "temperature": 0.2,
    "max_tokens": 220,
    "seed": 7,
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "risk_list",
        "strict": true,
        "schema": {
          "type": "object",
          "properties": {
            "risks": {"type": "array", "items": {"type": "string"}}
          },
          "required": ["risks"],
          "additionalProperties": false
        }
      }
    }
  }'

Accepted request fields

Field	Type	Status	Actual behavior
`model`	string	Optional	Requested model identifier. An MLX single-model server normalizes aliases and serves the active model.
`messages`	Message[]	Required	Roles: system, developer, user, assistant, or tool. Content may be text or multimodal parts.
`temperature`	number	Compatible	Sampling temperature; request value overrides the server default.
`top_p`	number	Compatible	Nucleus sampling threshold; request value overrides the server default.
`top_k`	integer	AFM extension	Restrict sampling to the highest-probability tokens.
`min_p`	number	AFM extension	Remove candidates below a probability floor relative to the leading token.
`max_tokens`	integer	Compatible	Primary output-token limit.
`max_completion_tokens`	integer	Compatible alias	Used only when max_tokens is absent.
`repetition_penalty`	number	AFM extension	Primary repetition penalty.
`repeat_penalty`	number	Accepted alias	Used only when repetition_penalty is absent.
`presence_penalty`	number	Compatible	Applied by the MLX generation path.
`frequency_penalty`	number	Parsed only	Accepted by the decoder but currently ignored.
`seed`	integer	Compatible	Reproducibility control, subject to backend and hardware determinism.
`stop`	string[]	Compatible	Merged with CLI --stop values and deduplicated.
`stream`	boolean	Compatible	Requests SSE when the server was launched with streaming enabled.
`stream_options.include_usage`	boolean	Compatible	AFM defaults this to true; false suppresses the separate final usage chunk.
`logprobs`	boolean	Compatible	Return token log probabilities.
`top_logprobs`	integer	Compatible	Bounded by --max-logprobs; the default server ceiling is 20.
`tools`	Tool[]	Compatible	Function tools with optional JSON Schema parameters and strict flag.
`tool_choice`	string \| object	Compatible	auto, none, required, or one named function.
`parallel_tool_calls`	boolean	Compatible	false limits the assistant turn to at most one emitted tool call.
`response_format`	object	Compatible	text, json_object, or json_schema; request value overrides --guided-json.
`chat_template_kwargs`	object	AFM extension	Values supplied to the model's Jinja chat template, such as enable_thinking.
`reasoning_effort`	string	AFM extension	DeepSeek control: low, high, or max; normalized into chat_template_kwargs.
`user`	string	Accepted	Decoded for client compatibility; it does not select a local user or security boundary.

Precedence and normalization

Inputs	Winner or merge rule
`max_tokens` vs `max_completion_tokens`	`max_tokens` wins. The second field is an accepted fallback.
`repetition_penalty` vs `repeat_penalty`	`repetition_penalty` wins.
Request sampling vs CLI defaults	A supplied request value wins; omitted fields inherit the server configuration or backend default.
Request `response_format` vs `--guided-json`	The request format wins. The server default is used only when the request omits the field.
Request `stop` vs CLI `--stop`	The sequences are merged in CLI-then-request order and deduplicated.
`reasoning_effort` and `chat_template_kwargs`	The top-level effort is inserted into the template object. A model encoder's explicit `enable_thinking=false` remains authoritative.

Message contract

Roles may be system, developer, user, assistant, or tool. A developer role is mapped to system behavior by AFM's compatibility layer. content may be a string, null for a tool-call assistant message, or an array of parts:

{"type":"text","text":"…"}
{"type":"image_url","image_url":{"url":"data:image/png;base64,…","detail":"high"}}
{"type":"input_audio","input_audio":{"data":"…","format":"wav","language":"en-US"}}

Multi-turn tool conversations send assistant tool_calls back verbatim, followed by one or more tool messages whose tool_call_id matches the call.

Non-streaming response

{
  "id": "chatcmpl-9a12bc34",
  "object": "chat.completion",
  "created": 1786128300,
  "model": "mlx-community/Qwen3.5-4B-4bit",
  "system_fingerprint": "afm_mlx__mlx-community__Qwen3.5-4B-4bit",
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "{\"risks\":[\"...\",\"...\"]}"
    },
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 41,
    "completion_tokens": 22,
    "total_tokens": 63,
    "completion_tokens_per_second": 38.4,
    "peak_memory_gib": 3.71
  }
}

choices[0].message.content is always present, including as JSON null when the assistant returns tool calls. Reasoning models can add reasoning_content. Tool turns add tool_calls and finish with finish_reason: "tool_calls".

Usage and AFM response extensions

Field	Meaning
`usage.prompt_tokens_details.cached_tokens`	Prompt tokens served from the prefix cache, when available.
`usage.prompt_time` / `completion_time` / `total_time`	Request timing in seconds.
`usage.prompt_tokens_per_second` / `completion_tokens_per_second`	Measured phase throughput.
`usage.peak_memory_gib`	Request-scoped MLX peak memory estimate.
`timings`	llama.cpp-style prompt and prediction counts and milliseconds.
`afm_profile` / `afm_profile_extended`	Optional MLX GPU, memory, and bandwidth measurements requested with `X-AFM-Profile`.

Backend boundaries

The Foundation backend accepts the shared request shape but has a narrower generation-control surface. MLX is the path for top_k, min_p, logprobs, model-native tools, grammar enforcement, cache statistics, and GPU profiling. Accepted JSON is therefore not the same thing as identical effect across backends.