AFM — local AI infrastructure for Apple Silicon

AFM streams OpenAI-shaped chat chunks over server-sent events. Correct clients preserve event order, assemble tool arguments by index, retain reasoning separately, and treat [DONE] as the transport terminator.

Request and transport

curl -N http://127.0.0.1:9999/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -H 'X-Request-ID: stream-8f31' \
  -d '{
    "messages": [{"role":"user","content":"Give me a two-line answer."}],
    "stream": true,
    "stream_options": {"include_usage": true}
  }'

Response header	Meaning
`Content-Type: text/event-stream`	The body is an SSE sequence.
`Cache-Control: no-cache`	Intermediaries should not cache the stream.
`X-Accel-Buffering: no`	Reverse proxies should forward events immediately.
`X-Request-ID` and `OpenAI-Request-ID`	The correlation and cancellation handle for this generation.
`X-Grammar-Constraints: downgraded`	A requested strict schema or eligible strict tool grammar is running best-effort.

Normal lifecycle

Role and first delta

The first content chunk can include delta.role: "assistant". Reasoning models may instead begin with reasoning_content.

Content or tool deltas

Append content and reasoning_content independently. Merge tool_calls by their numeric index.

Finish chunk

An empty delta carries finish_reason, commonly stop, length, tool_calls, or cancelled.

Usage chunk

When enabled, a separate chunk has an empty choices array and a top-level usage object.

Profile event

If requested, an AFM-only profile object is emitted after usage and before the terminator.

Transport terminator

The literal event data: [DONE] ends the logical stream; the HTTP body then closes.

data: {"id":"chatcmpl-…","object":"chat.completion.chunk","choices":[{"delta":{"role":"assistant","content":"First"},"finish_reason":null}]}

data: {"id":"chatcmpl-…","object":"chat.completion.chunk","choices":[{"delta":{"content":" line"},"finish_reason":null}]}

data: {"id":"chatcmpl-…","object":"chat.completion.chunk","choices":[{"delta":{},"finish_reason":"stop"}]}

data: {"id":"chatcmpl-…","object":"chat.completion.chunk","choices":[],"usage":{"prompt_tokens":18,"completion_tokens":9,"total_tokens":27}}

data: [DONE]

Streaming tool calls

The tool name and JSON argument string may be fragmented across arbitrary token boundaries. Store state per tool_calls[].index; keep the first non-null id, type, and function.name; append every function.arguments fragment; parse JSON only after the tool-call turn finishes.

data: {"choices":[{"delta":{"tool_calls":[{
  "index":0,
  "id":"call_…",
  "type":"function",
  "function":{"name":"read_file","arguments":"{\"pa"}
}]},"finish_reason":null}]}

data: {"choices":[{"delta":{"tool_calls":[{
  "index":0,
  "function":{"arguments":"th\":\"README.md\"}"}
}]},"finish_reason":null}]}

When parallel_tool_calls: false, AFM limits the assistant turn to at most one call. Otherwise multiple indices can advance interleaved.

Reasoning and structured output

Reasoning-capable models stream extracted thinking through delta.reasoning_content and the answer through delta.content. Keep them as distinct channels. For json_object and json_schema, AFM can defer or sanitize structured content so clients receive an unfenced JSON payload instead of markdown code fences.

Cancellation

curl -X POST http://127.0.0.1:9999/v1/chat/completions/stream-8f31/cancel

The path ID is the request correlation value, not the chatcmpl-… stream ID. A registered stream returns HTTP 200 with cancelled: true. Unknown, already completed, or expired IDs return 404 with cancelled: false. Cooperative cancellation ends the stream cleanly with finish_reason: "cancelled", then [DONE].

Optional profiling events

Request header	Response behavior
`X-AFM-Profile: true`	Final `{"afm_profile":{…}}` event with GPU power, memory, prefill/decode rates, chip, and estimated bandwidth when available.
`X-AFM-Profile: extended`	Final `{"afm_profile_extended":{"summary":{…},"samples":[…]}}` event with time-series samples.

These objects are AFM extensions and do not have a choices array. Consumers should dispatch on top-level keys before applying an OpenAI chunk decoder.

Mid-stream failure

Before headers are sent, failures use ordinary HTTP status codes and the JSON error envelope. After streaming begins, the status is already 200; a non-cancellation generation error may arrive as an error-marked content chunk before [DONE]. Production clients should therefore handle both HTTP errors and an abnormal stream payload.