AFM — local AI infrastructure for Apple Silicon

AFM uses HTTP status for transport-level outcome, an OpenAI-shaped JSON envelope for most typed API failures, and correlation headers across the server. The type or code tells software what happened; the message is for people and logs.

Error envelope

{
  "error": {
    "message": "tokenize endpoints require an MLX model — Foundation backend has no public tokenizer",
    "type": "tokenize_unsupported",
    "code": "tokenize_unsupported",
    "request_id": "req_8d132a7c90f1"
  }
}

Field	Guarantee
`error.message`	Human-readable explanation. Do not parse it to drive control flow.
`error.type`	Machine-oriented family such as `invalid_request_error`, `server_busy`, or `vision_unavailable`.
`error.code`	Optional stable code. Tokenizer middleware populates it; other controller paths may leave it `null`.
`error.request_id`	Optional correlation value. The response headers are the universal place to retrieve it.

Correlation contract

AFM honors the first non-empty inbound X-Request-ID or OpenAI-Request-ID. If neither exists, it mints req_ plus 12 lowercase UUID characters. Both header names are then placed on the response.

curl http://127.0.0.1:9999/v1/models \
  -H 'X-Request-ID: indexing-run-018' \
  -i

X-Request-ID: indexing-run-018
OpenAI-Request-ID: indexing-run-018

Status and recovery map

Status	Typical type or condition	Client action
`400`	`invalid_request_error`, malformed input, empty messages, unsupported field value, or `top_logprobs` above the configured ceiling.	Correct the request. Do not retry unchanged.
`404`	Unknown embedding model, file, batch, or cancellation ID.	Refresh discovery or local state. A cancellation 404 also means the request may already be complete.
`413`	`payload_too_large`.	Trim conversation history, reduce the file, or split the operation.
`422`	`tokenize_unsupported` when the Foundation backend has no public tokenizer.	Skip exact preflight counting, estimate conservatively, or run an MLX model.
`500`	Unexpected embedding or local framework failure.	Log the correlation ID; retry only if the operation is safe and the failure appears transient.
`501`	A requested Vision mode requires a newer macOS release.	Choose a supported mode or upgrade the host.
`503`	`server_busy`, `vision_unavailable`, `speech_unavailable`, or an embedding backend/asset availability failure.	Classify by type. Retry capacity pressure; change the environment for missing capabilities.

Capacity pressure

An MLX chat request waits for a concurrency slot for up to four minutes. If the queue still cannot admit it, AFM returns HTTP 503, type: "server_busy", and Retry-After: 2. A robust client:

honors the two-second minimum;
adds bounded exponential backoff and jitter across repeated failures;
keeps the same logical job ID but uses a fresh transport attempt ID if its tracing model distinguishes them;
caps retries and surfaces a useful human message;
uses /metrics to decide whether concurrency should be reduced.

Unavailable is not internal

Apple-native endpoints separate missing platform capability from malformed input. Speech on an unsupported OS produces speech_unavailable; Vision factory or framework absence produces vision_unavailable; unavailable embedding assets or backends return 503 under embedding_error. These responses tell an agent to change host capability or provisioning, not to rewrite an otherwise valid payload.

Best-effort strictness is a response signal

A strict JSON Schema or supported strict tool definition can be accepted even when the grammar engine is disabled. In that case the HTTP request succeeds, but AFM returns X-Grammar-Constraints: downgraded. Schema-sensitive clients should treat that header as a contract downgrade: validate the generated JSON themselves and decide whether to retry on a grammar-enabled server.

Errors after SSE begins

Once the server has sent HTTP 200 and stream headers, it cannot replace the response with a later 4xx or 5xx. A mid-generation error may appear as an error-marked assistant content chunk, followed by [DONE]. Cooperative cancellation is different: it emits a finish chunk with finish_reason: "cancelled" and no synthetic warning content.

Browser clients

AFM enables cross-origin access for local client UIs, but the accepted preflight headers vary by route. Embeddings reflect requested SDK headers; chat explicitly allows Content-Type, Authorization, and X-AFM-Profile; cancellation and token endpoints also name the correlation headers. Browser agents should perform normal preflight handling and avoid assuming one route's header allowlist applies globally.