oMLX

oMLX is a native macOS inference server built on Apple's MLX framework, packaged as a signed menu-bar app with a web dashboard. Its headline trick is paged SSD KV caching: a two-tier hot-RAM / cold-SSD architecture in safetensors format that persists previously seen prefixes across requests and even server restarts, so coding agents on long contexts cut TTFT from the typical 30-90 seconds you see in Ollama or LM Studio down to under five seconds. It also adds continuous batching via mlx-lm's BatchGenerator (up to ~4.14x speedup at 8x concurrency) and multi-model hosting with LRU eviction for LLMs, VLMs, embeddings and rerankers loaded at once.

The target user is obvious: developers running Claude Code, Cursor, OpenClaw or other agentic coding tools locally on an M-series Mac (ideally an M3 Ultra, though M1+/macOS 15+/64GB is the realistic floor). It exposes both OpenAI-compatible /v1/chat/completions and a native Anthropic /v1/messages endpoint, with a one-click config generator that emits the exact CLI command for each downstream tool. Tool calling is broad - JSON, Qwen, Gemma, GLM, MiniMax formats plus MCP - and reasoning models get automatic <think> tag handling.

oMLX is Apache 2.0 and free to download as a DMG or build from source; there is no paid tier advertised. It reuses an existing LM Studio model directory and has a built-in HuggingFace downloader, so onboarding is painless if you already run local models. Caveat: it is Apple Silicon only - no Linux, no Windows, no NVIDIA - and the published benchmarks are skewed toward a 512GB M3 Ultra, which is not what most readers actually own.