AFM

Model storage

Make checkpoint lookup and downloads predictable.

AFM accepts local paths and Hugging Face model IDs. Curated model lookup and Hugging Face downloads are separate concerns, so use the variable that matches the layout you actually manage.

Recommended setup

mkdir -p ~/.cache/afm-models ~/.cache/huggingface/hub
export MACAFM_MLX_MODEL_CACHE=~/.cache/afm-models
export HF_HUB_CACHE=~/.cache/huggingface/hub

MACAFM_MLX_MODEL_CACHE is a read-only lookup root for side-loaded checkpoints; AFM checks flat organization paths, a models/ subtree, and Hugging Face-style cache names below it. It does not redirect downloads. HF_HUB_CACHE identifies the standard Hub cache used for downloaded snapshots.

Lookup order

  1. MACAFM_MLX_MODEL_CACHE: flat, models/, and Hugging Face-style layouts.
  2. Swift Hub default: ~/Documents/huggingface/models/….
  3. HUGGINGFACE_HUB_CACHE or HF_HUB_CACHE.
  4. HF_HOME.
  5. XDG_CACHE_HOME.
  6. Python Hugging Face default: ~/.cache/huggingface/hub.
  7. Legacy locations below ~/Library/Caches.

A candidate is valid when it contains config.json and one or more .safetensors files. For Hugging Face cache layouts, AFM enters the resolved directory below snapshots/.

Model IDs and paths

An unqualified ID such as Qwen3-0.6B-4bit defaults to mlx-community/Qwen3-0.6B-4bit. Pass an organization-qualified ID when the model lives elsewhere, or pass an existing local directory when you need an exact checkpoint.

Capacity planning

Checkpoint size is only part of memory demand. Runtime weights, KV state, context length, concurrent slots, vision media, and speculative drafter artifacts all consume unified memory. Start with a small 0.6B–4B quantized model to verify installation before moving to 30B-class checkpoints.